2026-02-19T16:12:40.009 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-19T16:12:40.026 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T16:12:40.102 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59050 branch: wip-yuri-testing-2026-02-18-1820 description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '59050' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri-testing-2026-02-18-1820 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a workunit: branch: wip-yuri-testing-2026-02-18-1820 sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 7379 sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri-testing-2026-02-18-1820 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a targets: trial127.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNjcsax4DdqNFf+42DPNFA4oiUs7cjmRjHlYMEOm96/6o0nuaOg7pK/tfs0f7/VDjb50psiz9kKE1+n1brwhdF4= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-19_15:38:40 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-19T16:12:40.103 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa; will attempt to use it 2026-02-19T16:12:40.103 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa/tasks 2026-02-19T16:12:40.103 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-19T16:12:40.105 INFO:teuthology.task.internal:Checking packages... 2026-02-19T16:12:40.105 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '802bf111cd55e57f5842153d7ba2b8b964aa818a' 2026-02-19T16:12:40.105 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-19T16:12:40.105 INFO:teuthology.packaging:ref: None 2026-02-19T16:12:40.105 INFO:teuthology.packaging:tag: None 2026-02-19T16:12:40.105 INFO:teuthology.packaging:branch: wip-yuri-testing-2026-02-18-1820 2026-02-19T16:12:40.105 INFO:teuthology.packaging:sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T16:12:40.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri-testing-2026-02-18-1820 2026-02-19T16:12:40.225 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5336.g802bf111 2026-02-19T16:12:40.226 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-19T16:12:40.238 INFO:teuthology.task.internal:no buildpackages task found 2026-02-19T16:12:40.238 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-19T16:12:40.250 INFO:teuthology.task.internal:Saving configuration 2026-02-19T16:12:40.258 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-19T16:12:40.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-19T16:12:40.347 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial127.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59050', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-19 16:09:16.277659', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNjcsax4DdqNFf+42DPNFA4oiUs7cjmRjHlYMEOm96/6o0nuaOg7pK/tfs0f7/VDjb50psiz9kKE1+n1brwhdF4='} 2026-02-19T16:12:40.347 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-19T16:12:40.361 INFO:teuthology.task.internal:roles: ubuntu@trial127.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-19T16:12:40.361 INFO:teuthology.run_tasks:Running task console_log... 2026-02-19T16:12:40.444 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x712c7c9f7c40>, signals=[15]) 2026-02-19T16:12:40.444 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-19T16:12:40.453 INFO:teuthology.task.internal:Opening connections... 2026-02-19T16:12:40.454 DEBUG:teuthology.task.internal:connecting to ubuntu@trial127.front.sepia.ceph.com 2026-02-19T16:12:40.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial127.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T16:12:40.519 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-19T16:12:40.533 DEBUG:teuthology.orchestra.run.trial127:> uname -m 2026-02-19T16:12:40.608 INFO:teuthology.orchestra.run.trial127.stdout:x86_64 2026-02-19T16:12:40.609 DEBUG:teuthology.orchestra.run.trial127:> cat /etc/os-release 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:NAME="CentOS Stream" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:VERSION="9" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:ID="centos" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:ID_LIKE="rhel fedora" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:VERSION_ID="9" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:PLATFORM_ID="platform:el9" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:ANSI_COLOR="0;31" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:LOGO="fedora-logo-icon" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:HOME_URL="https://centos.org/" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-19T16:12:40.665 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-19T16:12:40.666 INFO:teuthology.lock.ops:Updating trial127.front.sepia.ceph.com on lock server 2026-02-19T16:12:40.733 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-19T16:12:40.744 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-19T16:12:40.755 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-19T16:12:40.755 DEBUG:teuthology.orchestra.run.trial127:> test '!' -e /home/ubuntu/cephtest 2026-02-19T16:12:40.770 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-19T16:12:40.782 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-19T16:12:40.783 DEBUG:teuthology.orchestra.run.trial127:> test -z $(ls -A /var/lib/ceph) 2026-02-19T16:12:40.854 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-19T16:12:40.948 INFO:teuthology.run_tasks:Running task kernel... 2026-02-19T16:12:40.971 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-19T16:12:40.971 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-19T16:12:40.971 DEBUG:teuthology.orchestra.run.trial127:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-19T16:12:40.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T16:12:40.988 DEBUG:teuthology.orchestra.run.trial127:> uname -r 2026-02-19T16:12:41.044 INFO:teuthology.orchestra.run.trial127.stdout:5.14.0-681.el9.x86_64 2026-02-19T16:12:41.044 INFO:teuthology.task.kernel:Running kernel on trial127: 5.14.0-681.el9.x86_64 2026-02-19T16:12:41.045 DEBUG:teuthology.orchestra.run.trial127:> sudo yum install -y kernel 2026-02-19T16:12:50.063 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:08 2026-02-19T16:13:17.274 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-19T16:13:22.249 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 7.8 MB 00:02 2026-02-19T16:13:23.245 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 20 kB 00:00 2026-02-19T16:13:23.643 INFO:teuthology.orchestra.run.trial127.stdout:Extra Packages for Enterprise Linux 60 MB/s | 20 MB 00:00 2026-02-19T16:13:26.859 INFO:teuthology.orchestra.run.trial127.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-19T16:13:27.879 INFO:teuthology.orchestra.run.trial127.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-19T16:13:27.880 INFO:teuthology.orchestra.run.trial127.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-19T16:13:27.898 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:13:27.898 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:13:27.899 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:13:27.923 DEBUG:teuthology.orchestra.run.trial127:> echo no | sudo yum reinstall kernel || true 2026-02-19T16:13:28.229 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:02 ago on Thu 19 Feb 2026 04:13:26 PM UTC. 2026-02-19T16:13:28.309 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:13:28.310 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:13:28.310 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-19T16:13:28.310 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:13:28.310 INFO:teuthology.orchestra.run.trial127.stdout:Reinstalling: 2026-02-19T16:13:28.310 INFO:teuthology.orchestra.run.trial127.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-19T16:13:28.310 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:13:28.311 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:13:28.311 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:13:28.311 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:13:28.311 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 803 k 2026-02-19T16:13:28.311 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 0 2026-02-19T16:13:28.311 INFO:teuthology.orchestra.run.trial127.stderr:Operation aborted. 2026-02-19T16:13:28.331 INFO:teuthology.orchestra.run.trial127.stdout:Is this ok [y/N]: 2026-02-19T16:13:28.332 DEBUG:teuthology.orchestra.run.trial127:> sudo yum reinstall -y kernel || true 2026-02-19T16:13:28.637 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:02 ago on Thu 19 Feb 2026 04:13:26 PM UTC. 2026-02-19T16:13:28.719 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:13:28.719 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:13:28.719 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-19T16:13:28.719 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout:Reinstalling: 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 803 k 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 0 2026-02-19T16:13:28.720 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:13:28.974 INFO:teuthology.orchestra.run.trial127.stdout:kernel-5.14.0-681.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2026-02-19T16:13:28.974 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:13:28.974 INFO:teuthology.orchestra.run.trial127.stdout:Total 3.1 MB/s | 803 kB 00:00 2026-02-19T16:13:28.984 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:13:29.017 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:13:29.017 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:13:29.022 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:13:29.022 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:13:29.046 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:13:29.049 INFO:teuthology.orchestra.run.trial127.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-19T16:13:29.049 INFO:teuthology.orchestra.run.trial127.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T16:13:29.155 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T16:13:29.155 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-19T16:13:29.208 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T16:13:29.208 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:13:29.208 INFO:teuthology.orchestra.run.trial127.stdout:Reinstalled: 2026-02-19T16:13:29.208 INFO:teuthology.orchestra.run.trial127.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-19T16:13:29.208 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:13:29.208 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:13:29.237 DEBUG:teuthology.orchestra.run.trial127:> rpm -q kernel | sort -rV | head -n 1 2026-02-19T16:13:29.259 INFO:teuthology.orchestra.run.trial127.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-19T16:13:29.260 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-19T16:13:29.260 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-19T16:13:29.260 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-19T16:13:29.260 DEBUG:teuthology.orchestra.run.trial127:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-19T16:13:29.324 INFO:teuthology.orchestra.run.trial127.stdout:ttyS1 2026-02-19T16:13:29.349 DEBUG:teuthology.parallel:result is None 2026-02-19T16:13:29.350 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-19T16:13:29.426 INFO:teuthology.task.internal:Creating test directory... 2026-02-19T16:13:29.427 DEBUG:teuthology.orchestra.run.trial127:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-19T16:13:29.443 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-19T16:13:29.455 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-19T16:13:29.470 INFO:teuthology.task.internal:Creating archive directory... 2026-02-19T16:13:29.470 DEBUG:teuthology.orchestra.run.trial127:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-19T16:13:29.510 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-19T16:13:29.524 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-19T16:13:29.524 DEBUG:teuthology.orchestra.run.trial127:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-19T16:13:29.565 INFO:teuthology.orchestra.run.trial127.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:13:29.573 INFO:teuthology.orchestra.run.trial127.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:13:29.575 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-19T16:13:29.586 INFO:teuthology.task.internal:Configuring sudo... 2026-02-19T16:13:29.586 DEBUG:teuthology.orchestra.run.trial127:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-19T16:13:29.640 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-19T16:13:29.659 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-19T16:13:29.660 DEBUG:teuthology.orchestra.run.trial127:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-19T16:13:29.694 DEBUG:teuthology.orchestra.run.trial127:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T16:13:29.858 DEBUG:teuthology.orchestra.run.trial127:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T16:13:29.882 DEBUG:teuthology.orchestra.run.trial127:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T16:13:30.038 DEBUG:teuthology.orchestra.run.trial127:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T16:13:30.063 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-19T16:13:30.063 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-19T16:13:30.127 DEBUG:teuthology.orchestra.run.trial127:> sudo service rsyslog restart 2026-02-19T16:13:30.195 INFO:teuthology.orchestra.run.trial127.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T16:13:30.589 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-19T16:13:30.600 INFO:teuthology.task.internal:Starting timer... 2026-02-19T16:13:30.600 INFO:teuthology.run_tasks:Running task pcp... 2026-02-19T16:13:30.630 INFO:teuthology.run_tasks:Running task selinux... 2026-02-19T16:13:30.653 DEBUG:teuthology.orchestra.run.trial127:> sudo service auditd rotate 2026-02-19T16:13:30.683 INFO:teuthology.orchestra.run.trial127.stdout:Rotating logs: 2026-02-19T16:13:30.684 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-19T16:13:30.684 DEBUG:teuthology.orchestra.run.trial127:> /usr/sbin/getenforce 2026-02-19T16:13:30.740 INFO:teuthology.orchestra.run.trial127.stdout:Permissive 2026-02-19T16:13:30.740 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial127.front.sepia.ceph.com': 'permissive'} 2026-02-19T16:13:30.740 DEBUG:teuthology.orchestra.run.trial127:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-19T16:13:30.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T16:13:30.806 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-19T16:13:30.806 DEBUG:teuthology.orchestra.run.trial127:> sudo /usr/sbin/setenforce permissive 2026-02-19T16:13:30.871 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-19T16:13:30.888 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-19T16:13:30.890 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-19T16:13:31.002 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-19T16:13:31.010 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-19T16:13:31.011 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial127.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-19T16:16:10.949 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial127.front.sepia.ceph.com')] 2026-02-19T16:16:10.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial127.front.sepia.ceph.com' 2026-02-19T16:16:10.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial127.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T16:16:11.066 DEBUG:teuthology.orchestra.run.trial127:> true 2026-02-19T16:16:11.139 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial127.front.sepia.ceph.com' 2026-02-19T16:16:11.140 INFO:teuthology.run_tasks:Running task clock... 2026-02-19T16:16:11.155 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-19T16:16:11.155 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-19T16:16:11.155 DEBUG:teuthology.orchestra.run.trial127:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T16:16:11.206 INFO:teuthology.orchestra.run.trial127.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-19T16:16:11.218 INFO:teuthology.orchestra.run.trial127.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-19T16:16:11.239 INFO:teuthology.orchestra.run.trial127.stderr:sudo: ntpd: command not found 2026-02-19T16:16:11.249 INFO:teuthology.orchestra.run.trial127.stdout:506 Cannot talk to daemon 2026-02-19T16:16:11.262 INFO:teuthology.orchestra.run.trial127.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-19T16:16:11.273 INFO:teuthology.orchestra.run.trial127.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-19T16:16:11.316 INFO:teuthology.orchestra.run.trial127.stderr:bash: line 1: ntpq: command not found 2026-02-19T16:16:11.353 INFO:teuthology.orchestra.run.trial127.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T16:16:11.353 INFO:teuthology.orchestra.run.trial127.stdout:=============================================================================== 2026-02-19T16:16:11.353 INFO:teuthology.orchestra.run.trial127.stdout:^? 149.28.200.179.vultruser> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-19T16:16:11.353 INFO:teuthology.orchestra.run.trial127.stdout:^? ntp.netlinkify.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-19T16:16:11.353 INFO:teuthology.orchestra.run.trial127.stdout:^? 38.70.62.230 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-19T16:16:11.353 INFO:teuthology.orchestra.run.trial127.stdout:^? 144.202.62.209.vultruser> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-19T16:16:11.354 INFO:teuthology.run_tasks:Running task install... 2026-02-19T16:16:11.368 DEBUG:teuthology.task.install:project ceph 2026-02-19T16:16:11.368 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}} 2026-02-19T16:16:11.368 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'} 2026-02-19T16:16:11.368 INFO:teuthology.task.install:Using flavor: default 2026-02-19T16:16:11.379 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-19T16:16:11.379 INFO:teuthology.task.install:extra packages: [] 2026-02-19T16:16:11.379 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a', 'tag': None, 'wait_for_package': False} 2026-02-19T16:16:11.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T16:16:25.398 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/ 2026-02-19T16:16:25.398 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5336.g802bf111 2026-02-19T16:16:25.682 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-19T16:16:25.682 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-19T16:16:25.682 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-19T16:16:25.708 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-19T16:16:25.709 DEBUG:teuthology.orchestra.run.trial127:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/802bf111cd55e57f5842153d7ba2b8b964aa818a/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-19T16:16:25.774 DEBUG:teuthology.orchestra.run.trial127:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-19T16:16:25.849 DEBUG:teuthology.orchestra.run.trial127:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-19T16:16:25.912 INFO:teuthology.orchestra.run.trial127.stdout:check_obsoletes = 1 2026-02-19T16:16:25.914 DEBUG:teuthology.orchestra.run.trial127:> sudo yum clean all 2026-02-19T16:16:26.075 INFO:teuthology.orchestra.run.trial127.stdout:41 files removed 2026-02-19T16:16:26.091 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-radosgw 2026-02-19T16:16:26.877 INFO:teuthology.orchestra.run.trial127.stdout:ceph packages for x86_64 146 kB/s | 93 kB 00:00 2026-02-19T16:16:27.449 INFO:teuthology.orchestra.run.trial127.stdout:ceph noarch packages 45 kB/s | 24 kB 00:00 2026-02-19T16:16:27.932 INFO:teuthology.orchestra.run.trial127.stdout:ceph source packages 4.5 kB/s | 2.1 kB 00:00 2026-02-19T16:16:29.013 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.9 MB 00:01 2026-02-19T16:16:31.766 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - AppStream 12 MB/s | 27 MB 00:02 2026-02-19T16:16:35.196 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 7.8 MB 00:01 2026-02-19T16:16:36.970 INFO:teuthology.orchestra.run.trial127.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 20 kB 00:01 2026-02-19T16:17:02.347 INFO:teuthology.orchestra.run.trial127.stdout:Extra Packages for Enterprise Linux 820 kB/s | 20 MB 00:25 2026-02-19T16:17:09.918 INFO:teuthology.orchestra.run.trial127.stdout:lab-extras 11 kB/s | 50 kB 00:04 2026-02-19T16:17:11.118 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repository Size 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: ceph-radosgw x86_64 2:20.3.0-5336.g802bf111.el9 ceph 13 M 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout:Upgrading: 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: librados2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 3.6 M 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: librbd1 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.8 M 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout:Installing dependencies: 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: ceph-base x86_64 2:20.3.0-5336.g802bf111.el9 ceph 5.2 M 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: ceph-common x86_64 2:20.3.0-5336.g802bf111.el9 ceph 24 M 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: ceph-selinux x86_64 2:20.3.0-5336.g802bf111.el9 ceph 25 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 910 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: libradosstriper1 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 250 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: librgw2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 6.5 M 2026-02-19T16:17:11.120 INFO:teuthology.orchestra.run.trial127.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-argparse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 45 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-common x86_64 2:20.3.0-5336.g802bf111.el9 ceph 185 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: python3-cephfs x86_64 2:20.3.0-5336.g802bf111.el9 ceph 203 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: python3-rados x86_64 2:20.3.0-5336.g802bf111.el9 ceph 323 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: python3-rbd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 305 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: python3-rgw x86_64 2:20.3.0-5336.g802bf111.el9 ceph 100 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout:Installing weak dependencies: 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-daemon x86_64 2:20.3.0-5336.g802bf111.el9 ceph 37 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 25 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout:Install 35 Packages 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout:Upgrade 2 Packages 2026-02-19T16:17:11.121 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:11.122 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 68 M 2026-02-19T16:17:11.122 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:17:14.395 INFO:teuthology.orchestra.run.trial127.stdout:(1/37): ceph-base-20.3.0-5336.g802bf111.el9.x86 2.9 MB/s | 5.2 MB 00:01 2026-02-19T16:17:14.444 INFO:teuthology.orchestra.run.trial127.stdout:(2/37): ceph-selinux-20.3.0-5336.g802bf111.el9. 515 kB/s | 25 kB 00:00 2026-02-19T16:17:14.495 INFO:teuthology.orchestra.run.trial127.stdout:(3/37): libcephfs-daemon-20.3.0-5336.g802bf111. 731 kB/s | 37 kB 00:00 2026-02-19T16:17:14.549 INFO:teuthology.orchestra.run.trial127.stdout:(4/37): libcephfs-proxy2-20.3.0-5336.g802bf111. 469 kB/s | 25 kB 00:00 2026-02-19T16:17:14.861 INFO:teuthology.orchestra.run.trial127.stdout:(5/37): libcephfs2-20.3.0-5336.g802bf111.el9.x8 2.8 MB/s | 910 kB 00:00 2026-02-19T16:17:14.991 INFO:teuthology.orchestra.run.trial127.stdout:(6/37): libradosstriper1-20.3.0-5336.g802bf111. 1.9 MB/s | 250 kB 00:00 2026-02-19T16:17:15.366 INFO:teuthology.orchestra.run.trial127.stdout:(7/37): ceph-radosgw-20.3.0-5336.g802bf111.el9. 4.8 MB/s | 13 MB 00:02 2026-02-19T16:17:15.427 INFO:teuthology.orchestra.run.trial127.stdout:(8/37): python3-ceph-argparse-20.3.0-5336.g802b 738 kB/s | 45 kB 00:00 2026-02-19T16:17:15.490 INFO:teuthology.orchestra.run.trial127.stdout:(9/37): python3-ceph-common-20.3.0-5336.g802bf1 2.9 MB/s | 185 kB 00:00 2026-02-19T16:17:15.554 INFO:teuthology.orchestra.run.trial127.stdout:(10/37): python3-cephfs-20.3.0-5336.g802bf111.e 3.1 MB/s | 203 kB 00:00 2026-02-19T16:17:15.623 INFO:teuthology.orchestra.run.trial127.stdout:(11/37): python3-rados-20.3.0-5336.g802bf111.el 4.6 MB/s | 323 kB 00:00 2026-02-19T16:17:15.692 INFO:teuthology.orchestra.run.trial127.stdout:(12/37): python3-rbd-20.3.0-5336.g802bf111.el9. 4.3 MB/s | 305 kB 00:00 2026-02-19T16:17:15.760 INFO:teuthology.orchestra.run.trial127.stdout:(13/37): python3-rgw-20.3.0-5336.g802bf111.el9. 1.4 MB/s | 100 kB 00:00 2026-02-19T16:17:16.125 INFO:teuthology.orchestra.run.trial127.stdout:(14/37): ceph-common-20.3.0-5336.g802bf111.el9. 6.8 MB/s | 24 MB 00:03 2026-02-19T16:17:16.204 INFO:teuthology.orchestra.run.trial127.stdout:(15/37): mailcap-2.1.49-5.el9.noarch.rpm 75 kB/s | 33 kB 00:00 2026-02-19T16:17:16.384 INFO:teuthology.orchestra.run.trial127.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 519 kB/s | 93 kB 00:00 2026-02-19T16:17:16.515 INFO:teuthology.orchestra.run.trial127.stdout:(17/37): librgw2-20.3.0-5336.g802bf111.el9.x86_ 4.2 MB/s | 6.5 MB 00:01 2026-02-19T16:17:16.701 INFO:teuthology.orchestra.run.trial127.stdout:(18/37): boost-program-options-1.75.0-13.el9.x8 328 kB/s | 104 kB 00:00 2026-02-19T16:17:17.016 INFO:teuthology.orchestra.run.trial127.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 507 kB/s | 160 kB 00:00 2026-02-19T16:17:17.176 INFO:teuthology.orchestra.run.trial127.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 283 kB/s | 45 kB 00:00 2026-02-19T16:17:17.309 INFO:teuthology.orchestra.run.trial127.stdout:(21/37): libnbd-1.20.3-4.el9.x86_64.rpm 206 kB/s | 164 kB 00:00 2026-02-19T16:17:17.783 INFO:teuthology.orchestra.run.trial127.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 616 kB/s | 292 kB 00:00 2026-02-19T16:17:17.875 INFO:teuthology.orchestra.run.trial127.stdout:(23/37): nvme-cli-2.16-1.el9.x86_64.rpm 673 kB/s | 1.2 MB 00:01 2026-02-19T16:17:18.034 INFO:teuthology.orchestra.run.trial127.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 234 kB/s | 37 kB 00:00 2026-02-19T16:17:18.042 INFO:teuthology.orchestra.run.trial127.stdout:(25/37): librdkafka-1.6.1-102.el9.x86_64.rpm 766 kB/s | 662 kB 00:00 2026-02-19T16:17:18.178 INFO:teuthology.orchestra.run.trial127.stdout:(26/37): qatlib-25.08.0-2.el9.x86_64.rpm 606 kB/s | 240 kB 00:00 2026-02-19T16:17:18.193 INFO:teuthology.orchestra.run.trial127.stdout:[MIRROR] qatzip-libs-1.3.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/qatzip-libs-1.3.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-19T16:17:18.594 INFO:teuthology.orchestra.run.trial127.stdout:[MIRROR] qatzip-libs-1.3.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/qatzip-libs-1.3.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-19T16:17:18.703 INFO:teuthology.orchestra.run.trial127.stdout:(27/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 99 kB/s | 66 kB 00:00 2026-02-19T16:17:23.281 INFO:teuthology.orchestra.run.trial127.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 5.4 kB/s | 25 kB 00:04 2026-02-19T16:17:23.327 INFO:teuthology.orchestra.run.trial127.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2026-02-19T16:17:23.360 INFO:teuthology.orchestra.run.trial127.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2026-02-19T16:17:23.363 INFO:teuthology.orchestra.run.trial127.stdout:(31/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 58 kB/s | 308 kB 00:05 2026-02-19T16:17:23.395 INFO:teuthology.orchestra.run.trial127.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 5.8 MB/s | 191 kB 00:00 2026-02-19T16:17:23.456 INFO:teuthology.orchestra.run.trial127.stdout:(33/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 8.5 MB/s | 838 kB 00:00 2026-02-19T16:17:23.479 INFO:teuthology.orchestra.run.trial127.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2026-02-19T16:17:23.490 INFO:teuthology.orchestra.run.trial127.stdout:(35/37): libarrow-9.0.0-15.el9.x86_64.rpm 850 kB/s | 4.4 MB 00:05 2026-02-19T16:17:23.614 INFO:teuthology.orchestra.run.trial127.stdout:(36/37): librbd1-20.3.0-5336.g802bf111.el9.x86_ 21 MB/s | 2.8 MB 00:00 2026-02-19T16:17:23.677 INFO:teuthology.orchestra.run.trial127.stdout:(37/37): librados2-20.3.0-5336.g802bf111.el9.x8 16 MB/s | 3.6 MB 00:00 2026-02-19T16:17:23.679 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:17:23.680 INFO:teuthology.orchestra.run.trial127.stdout:Total 5.4 MB/s | 68 MB 00:12 2026-02-19T16:17:23.775 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:17:23.794 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:17:23.794 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:17:24.065 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:17:24.066 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:17:24.376 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:17:24.536 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-19T16:17:24.541 INFO:teuthology.orchestra.run.trial127.stdout: Upgrading : librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T16:17:24.592 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T16:17:24.593 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T16:17:24.605 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T16:17:24.607 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_6 4/39 2026-02-19T16:17:24.608 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T16:17:24.635 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T16:17:24.639 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 6/39 2026-02-19T16:17:24.647 INFO:teuthology.orchestra.run.trial127.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-19T16:17:24.649 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-19T16:17:24.654 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-19T16:17:24.658 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-19T16:17:24.669 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9. 11/39 2026-02-19T16:17:24.681 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 12/39 2026-02-19T16:17:24.682 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 13/39 2026-02-19T16:17:24.702 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 13/39 2026-02-19T16:17:24.706 INFO:teuthology.orchestra.run.trial127.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-19T16:17:24.735 INFO:teuthology.orchestra.run.trial127.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-19T16:17:24.739 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-19T16:17:24.745 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-19T16:17:24.747 INFO:teuthology.orchestra.run.trial127.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-19T16:17:24.753 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-19T16:17:24.877 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-19T16:17:24.878 INFO:teuthology.orchestra.run.trial127.stdout: Upgrading : librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 21/39 2026-02-19T16:17:24.907 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 21/39 2026-02-19T16:17:24.909 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 22/39 2026-02-19T16:17:24.917 INFO:teuthology.orchestra.run.trial127.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-19T16:17:25.208 INFO:teuthology.orchestra.run.trial127.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-19T16:17:25.209 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 25/39 2026-02-19T16:17:25.226 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 25/39 2026-02-19T16:17:25.231 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 26/39 2026-02-19T16:17:25.235 INFO:teuthology.orchestra.run.trial127.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-19T16:17:25.238 INFO:teuthology.orchestra.run.trial127.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-19T16:17:25.239 INFO:teuthology.orchestra.run.trial127.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-19T16:17:25.257 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-19T16:17:25.374 INFO:teuthology.orchestra.run.trial127.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-19T16:17:25.376 INFO:teuthology.orchestra.run.trial127.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-19T16:17:25.660 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-19T16:17:25.660 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-19T16:17:25.660 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:25.704 INFO:teuthology.orchestra.run.trial127.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-19T16:17:25.706 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x8 33/39 2026-02-19T16:17:26.818 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T16:17:26.820 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T16:17:27.085 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T16:17:27.088 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 35/39 2026-02-19T16:17:27.129 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 35/39 2026-02-19T16:17:27.129 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-19T16:17:27.129 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-19T16:17:27.129 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:27.132 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T16:17:31.712 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /sys 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /proc 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /mnt 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /var/tmp 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /home 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /root 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout:skipping the directory /tmp 2026-02-19T16:17:31.713 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:32.306 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 37/39 2026-02-19T16:17:32.330 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 37/39 2026-02-19T16:17:32.330 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:17:32.330 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-19T16:17:32.330 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-19T16:17:32.330 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-19T16:17:32.330 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:32.332 INFO:teuthology.orchestra.run.trial127.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-19T16:17:32.344 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-19T16:17:32.344 INFO:teuthology.orchestra.run.trial127.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T16:17:32.399 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T16:17:32.400 INFO:teuthology.orchestra.run.trial127.stdout:Creating group 'qat' with GID 992. 2026-02-19T16:17:32.400 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 1/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 4/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 6/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 7/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 8/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 9/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9. 10/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x8 11/39 2026-02-19T16:17:32.694 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 12/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 13/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 14/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 15/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librados2-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T16:17:32.695 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-19T16:17:32.696 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 38/39 2026-02-19T16:17:32.757 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T16:17:32.757 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:32.757 INFO:teuthology.orchestra.run.trial127.stdout:Upgraded: 2026-02-19T16:17:32.757 INFO:teuthology.orchestra.run.trial127.stdout: librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.758 INFO:teuthology.orchestra.run.trial127.stdout: python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: re2-1:20211101-20.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:32.759 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:17:32.800 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-test 2026-02-19T16:17:33.115 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:24 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:17:33.199 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repository Size 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: ceph-test x86_64 2:20.3.0-5336.g802bf111.el9 ceph 79 M 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:Installing dependencies: 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: libcephsqlite x86_64 2:20.3.0-5336.g802bf111.el9 ceph 159 k 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:Install 10 Packages 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 83 M 2026-02-19T16:17:33.200 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 341 M 2026-02-19T16:17:33.201 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:17:33.924 INFO:teuthology.orchestra.run.trial127.stdout:(1/10): libcephsqlite-20.3.0-5336.g802bf111.el9 345 kB/s | 159 kB 00:00 2026-02-19T16:17:34.171 INFO:teuthology.orchestra.run.trial127.stdout:(2/10): libatomic-11.5.0-14.el9.x86_64.rpm 92 kB/s | 23 kB 00:00 2026-02-19T16:17:34.210 INFO:teuthology.orchestra.run.trial127.stdout:(3/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.8 MB/s | 2.1 MB 00:00 2026-02-19T16:17:34.408 INFO:teuthology.orchestra.run.trial127.stdout:(4/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 396 kB/s | 94 kB 00:00 2026-02-19T16:17:39.064 INFO:teuthology.orchestra.run.trial127.stdout:(5/10): libxslt-1.1.34-12.el9.x86_64.rpm 48 kB/s | 233 kB 00:04 2026-02-19T16:17:39.115 INFO:teuthology.orchestra.run.trial127.stdout:(6/10): protobuf-3.14.0-17.el9.x86_64.rpm 218 kB/s | 1.0 MB 00:04 2026-02-19T16:17:39.117 INFO:teuthology.orchestra.run.trial127.stdout:(7/10): socat-1.7.4.1-8.el9.x86_64.rpm 5.6 MB/s | 303 kB 00:00 2026-02-19T16:17:39.141 INFO:teuthology.orchestra.run.trial127.stdout:(8/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.4 MB/s | 64 kB 00:00 2026-02-19T16:17:39.186 INFO:teuthology.orchestra.run.trial127.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.7 MB/s | 122 kB 00:00 2026-02-19T16:18:00.014 INFO:teuthology.orchestra.run.trial127.stdout:(10/10): ceph-test-20.3.0-5336.g802bf111.el9.x8 3.0 MB/s | 79 MB 00:26 2026-02-19T16:18:00.016 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:00.016 INFO:teuthology.orchestra.run.trial127.stdout:Total 3.1 MB/s | 83 MB 00:26 2026-02-19T16:18:00.079 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:00.090 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:00.090 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:00.375 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:00.375 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:00.686 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:00.694 INFO:teuthology.orchestra.run.trial127.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-19T16:18:00.703 INFO:teuthology.orchestra.run.trial127.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-19T16:18:00.708 INFO:teuthology.orchestra.run.trial127.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-19T16:18:00.712 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-19T16:18:00.718 INFO:teuthology.orchestra.run.trial127.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-19T16:18:00.720 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-19T16:18:00.730 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-19T16:18:00.738 INFO:teuthology.orchestra.run.trial127.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-19T16:18:00.739 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 9/10 2026-02-19T16:18:04.420 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 9/10 2026-02-19T16:18:04.438 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 10/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 10/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 1/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 2/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-19T16:18:04.785 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-19T16:18:04.840 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:04.841 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:04.882 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph 2026-02-19T16:18:05.190 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:00:56 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:05.278 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:05.280 INFO:teuthology.orchestra.run.trial127.stdout:======================================================================================= 2026-02-19T16:18:05.280 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repository Size 2026-02-19T16:18:05.280 INFO:teuthology.orchestra.run.trial127.stdout:======================================================================================= 2026-02-19T16:18:05.280 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph x86_64 2:20.3.0-5336.g802bf111.el9 ceph 6.5 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout:Installing dependencies: 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 57 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mds x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.4 M 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr x86_64 2:20.3.0-5336.g802bf111.el9 ceph 935 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 297 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mon x86_64 2:20.3.0-5336.g802bf111.el9 ceph 5.0 M 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 168 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-classic x86_64 2:20.3.0-5336.g802bf111.el9 ceph 16 M 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-crimson x86_64 2:20.3.0-5336.g802bf111.el9 ceph 19 M 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 17 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: cephadm noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 1.0 M 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-19T16:18:05.281 INFO:teuthology.orchestra.run.trial127.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-19T16:18:05.282 INFO:teuthology.orchestra.run.trial127.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-19T16:18:05.283 INFO:teuthology.orchestra.run.trial127.stdout:Installing weak dependencies: 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 184 k 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 51 M 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 7.4 M 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 22 k 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-rook noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 50 k 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: ceph-volume noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 301 k 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout:======================================================================================= 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout:Install 96 Packages 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 159 M 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 733 M 2026-02-19T16:18:05.284 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:10.181 INFO:teuthology.orchestra.run.trial127.stdout:(1/96): ceph-20.3.0-5336.g802bf111.el9.x86_64.r 1.5 kB/s | 6.5 kB 00:04 2026-02-19T16:18:10.759 INFO:teuthology.orchestra.run.trial127.stdout:(2/96): ceph-mgr-20.3.0-5336.g802bf111.el9.x86_ 191 kB/s | 935 kB 00:04 2026-02-19T16:18:10.876 INFO:teuthology.orchestra.run.trial127.stdout:(3/96): ceph-osd-20.3.0-5336.g802bf111.el9.x86_ 1.4 MB/s | 168 kB 00:00 2026-02-19T16:18:10.978 INFO:teuthology.orchestra.run.trial127.stdout:(4/96): ceph-mds-20.3.0-5336.g802bf111.el9.x86_ 476 kB/s | 2.4 MB 00:05 2026-02-19T16:18:13.650 INFO:teuthology.orchestra.run.trial127.stdout:(5/96): ceph-mon-20.3.0-5336.g802bf111.el9.x86_ 1.4 MB/s | 5.0 MB 00:03 2026-02-19T16:18:13.716 INFO:teuthology.orchestra.run.trial127.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5336.g80 864 kB/s | 57 kB 00:00 2026-02-19T16:18:13.851 INFO:teuthology.orchestra.run.trial127.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5336.g802bf111. 1.3 MB/s | 184 kB 00:00 2026-02-19T16:18:15.137 INFO:teuthology.orchestra.run.trial127.stdout:(8/96): ceph-osd-classic-20.3.0-5336.g802bf111. 3.7 MB/s | 16 MB 00:04 2026-02-19T16:18:15.412 INFO:teuthology.orchestra.run.trial127.stdout:(9/96): ceph-osd-crimson-20.3.0-5336.g802bf111. 4.3 MB/s | 19 MB 00:04 2026-02-19T16:18:15.476 INFO:teuthology.orchestra.run.trial127.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5336.g802bf1 350 kB/s | 22 kB 00:00 2026-02-19T16:18:15.609 INFO:teuthology.orchestra.run.trial127.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5336.g802 2.2 MB/s | 297 kB 00:00 2026-02-19T16:18:15.674 INFO:teuthology.orchestra.run.trial127.stdout:(12/96): ceph-mgr-rook-20.3.0-5336.g802bf111.el 768 kB/s | 50 kB 00:00 2026-02-19T16:18:15.740 INFO:teuthology.orchestra.run.trial127.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5336.g80 267 kB/s | 17 kB 00:00 2026-02-19T16:18:15.874 INFO:teuthology.orchestra.run.trial127.stdout:(14/96): ceph-volume-20.3.0-5336.g802bf111.el9. 2.2 MB/s | 301 kB 00:00 2026-02-19T16:18:16.154 INFO:teuthology.orchestra.run.trial127.stdout:(15/96): cephadm-20.3.0-5336.g802bf111.el9.noar 3.6 MB/s | 1.0 MB 00:00 2026-02-19T16:18:16.239 INFO:teuthology.orchestra.run.trial127.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 4.0 MB/s | 351 kB 00:00 2026-02-19T16:18:16.272 INFO:teuthology.orchestra.run.trial127.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.2 MB/s | 40 kB 00:00 2026-02-19T16:18:16.301 INFO:teuthology.orchestra.run.trial127.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 2.4 MB/s | 72 kB 00:00 2026-02-19T16:18:16.339 INFO:teuthology.orchestra.run.trial127.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 20 MB/s | 794 kB 00:00 2026-02-19T16:18:16.370 INFO:teuthology.orchestra.run.trial127.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 5.9 MB/s | 184 kB 00:00 2026-02-19T16:18:16.403 INFO:teuthology.orchestra.run.trial127.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 7.4 MB/s | 253 kB 00:00 2026-02-19T16:18:16.441 INFO:teuthology.orchestra.run.trial127.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 33 MB/s | 1.2 MB 00:00 2026-02-19T16:18:16.470 INFO:teuthology.orchestra.run.trial127.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 3.5 MB/s | 106 kB 00:00 2026-02-19T16:18:16.586 INFO:teuthology.orchestra.run.trial127.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 1.1 MB/s | 135 kB 00:00 2026-02-19T16:18:16.615 INFO:teuthology.orchestra.run.trial127.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 5.1 MB/s | 150 kB 00:00 2026-02-19T16:18:16.647 INFO:teuthology.orchestra.run.trial127.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 3.9 MB/s | 126 kB 00:00 2026-02-19T16:18:16.678 INFO:teuthology.orchestra.run.trial127.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 6.7 MB/s | 218 kB 00:00 2026-02-19T16:18:16.710 INFO:teuthology.orchestra.run.trial127.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 5.6 MB/s | 182 kB 00:00 2026-02-19T16:18:16.740 INFO:teuthology.orchestra.run.trial127.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 8.8 MB/s | 266 kB 00:00 2026-02-19T16:18:17.527 INFO:teuthology.orchestra.run.trial127.stdout:(30/96): ceph-mgr-diskprediction-local-20.3.0-5 3.1 MB/s | 7.4 MB 00:02 2026-02-19T16:18:18.309 INFO:teuthology.orchestra.run.trial127.stdout:(31/96): flexiblas-3.0.4-9.el9.x86_64.rpm 19 kB/s | 30 kB 00:01 2026-02-19T16:18:18.410 INFO:teuthology.orchestra.run.trial127.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 148 kB/s | 15 kB 00:00 2026-02-19T16:18:18.732 INFO:teuthology.orchestra.run.trial127.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 765 kB/s | 246 kB 00:00 2026-02-19T16:18:18.835 INFO:teuthology.orchestra.run.trial127.stdout:(34/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 2.3 MB/s | 3.0 MB 00:01 2026-02-19T16:18:18.849 INFO:teuthology.orchestra.run.trial127.stdout:(35/96): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2026-02-19T16:18:18.914 INFO:teuthology.orchestra.run.trial127.stdout:(36/96): openblas-0.3.29-1.el9.x86_64.rpm 532 kB/s | 42 kB 00:00 2026-02-19T16:18:19.135 INFO:teuthology.orchestra.run.trial127.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2026-02-19T16:18:19.217 INFO:teuthology.orchestra.run.trial127.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2026-02-19T16:18:19.283 INFO:teuthology.orchestra.run.trial127.stdout:(39/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 12 MB/s | 5.3 MB 00:00 2026-02-19T16:18:19.299 INFO:teuthology.orchestra.run.trial127.stdout:(40/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 3.0 MB/s | 249 kB 00:00 2026-02-19T16:18:19.363 INFO:teuthology.orchestra.run.trial127.stdout:(41/96): python3-jmespath-1.0.1-1.el9.noarch.rp 596 kB/s | 48 kB 00:00 2026-02-19T16:18:19.380 INFO:teuthology.orchestra.run.trial127.stdout:(42/96): python3-libstoragemgmt-1.10.1-1.el9.x8 2.1 MB/s | 177 kB 00:00 2026-02-19T16:18:19.455 INFO:teuthology.orchestra.run.trial127.stdout:(43/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2026-02-19T16:18:19.459 INFO:teuthology.orchestra.run.trial127.stdout:(44/96): python3-markupsafe-1.1.1-12.el9.x86_64 439 kB/s | 35 kB 00:00 2026-02-19T16:18:19.542 INFO:teuthology.orchestra.run.trial127.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 5.2 MB/s | 442 kB 00:00 2026-02-19T16:18:19.622 INFO:teuthology.orchestra.run.trial127.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 963 kB/s | 77 kB 00:00 2026-02-19T16:18:19.653 INFO:teuthology.orchestra.run.trial127.stdout:(47/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 31 MB/s | 6.1 MB 00:00 2026-02-19T16:18:19.703 INFO:teuthology.orchestra.run.trial127.stdout:(48/96): python3-protobuf-3.14.0-17.el9.noarch. 3.2 MB/s | 267 kB 00:00 2026-02-19T16:18:19.735 INFO:teuthology.orchestra.run.trial127.stdout:(49/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2026-02-19T16:18:19.785 INFO:teuthology.orchestra.run.trial127.stdout:(50/96): python3-pyasn1-modules-0.4.8-6.el9.noa 3.3 MB/s | 279 kB 00:00 2026-02-19T16:18:19.815 INFO:teuthology.orchestra.run.trial127.stdout:(51/96): python3-requests-oauthlib-1.3.0-12.el9 669 kB/s | 54 kB 00:00 2026-02-19T16:18:19.895 INFO:teuthology.orchestra.run.trial127.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 524 kB/s | 42 kB 00:00 2026-02-19T16:18:19.974 INFO:teuthology.orchestra.run.trial127.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.3 MB/s | 189 kB 00:00 2026-02-19T16:18:20.054 INFO:teuthology.orchestra.run.trial127.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.1 MB/s | 90 kB 00:00 2026-02-19T16:18:20.382 INFO:teuthology.orchestra.run.trial127.stdout:(55/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 32 MB/s | 19 MB 00:00 2026-02-19T16:18:20.463 INFO:teuthology.orchestra.run.trial127.stdout:(56/96): lua-devel-5.4.4-4.el9.x86_64.rpm 55 kB/s | 22 kB 00:00 2026-02-19T16:18:21.806 INFO:teuthology.orchestra.run.trial127.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 605 kB/s | 862 kB 00:01 2026-02-19T16:18:26.961 INFO:teuthology.orchestra.run.trial127.stdout:(58/96): ceph-mgr-dashboard-20.3.0-5336.g802bf1 3.9 MB/s | 51 MB 00:13 2026-02-19T16:18:26.963 INFO:teuthology.orchestra.run.trial127.stdout:(59/96): grpc-data-1.46.7-10.el9.noarch.rpm 3.8 kB/s | 19 kB 00:05 2026-02-19T16:18:27.049 INFO:teuthology.orchestra.run.trial127.stdout:(60/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 84 kB/s | 551 kB 00:06 2026-02-19T16:18:27.072 INFO:teuthology.orchestra.run.trial127.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 1.3 MB/s | 29 kB 00:00 2026-02-19T16:18:27.096 INFO:teuthology.orchestra.run.trial127.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 2.5 MB/s | 60 kB 00:00 2026-02-19T16:18:27.103 INFO:teuthology.orchestra.run.trial127.stdout:(63/96): python3-asyncssh-2.13.2-5.el9.noarch.r 3.8 MB/s | 548 kB 00:00 2026-02-19T16:18:27.107 INFO:teuthology.orchestra.run.trial127.stdout:(64/96): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2026-02-19T16:18:27.119 INFO:teuthology.orchestra.run.trial127.stdout:(65/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.8 MB/s | 43 kB 00:00 2026-02-19T16:18:27.130 INFO:teuthology.orchestra.run.trial127.stdout:(66/96): python3-cachetools-4.2.4-1.el9.noarch. 1.2 MB/s | 32 kB 00:00 2026-02-19T16:18:27.130 INFO:teuthology.orchestra.run.trial127.stdout:(67/96): python3-certifi-2023.05.07-4.el9.noarc 610 kB/s | 14 kB 00:00 2026-02-19T16:18:27.147 INFO:teuthology.orchestra.run.trial127.stdout:(68/96): python3-cheroot-10.0.1-4.el9.noarch.rp 6.1 MB/s | 173 kB 00:00 2026-02-19T16:18:27.167 INFO:teuthology.orchestra.run.trial127.stdout:(69/96): python3-cherrypy-18.6.1-2.el9.noarch.r 9.3 MB/s | 358 kB 00:00 2026-02-19T16:18:27.180 INFO:teuthology.orchestra.run.trial127.stdout:(70/96): python3-google-auth-2.45.0-1.el9.noarc 4.9 MB/s | 254 kB 00:00 2026-02-19T16:18:27.198 INFO:teuthology.orchestra.run.trial127.stdout:(71/96): python3-grpcio-tools-1.46.7-10.el9.x86 4.6 MB/s | 144 kB 00:00 2026-02-19T16:18:27.208 INFO:teuthology.orchestra.run.trial127.stdout:(72/96): python3-influxdb-5.3.1-1.el9.noarch.rp 5.0 MB/s | 139 kB 00:00 2026-02-19T16:18:27.222 INFO:teuthology.orchestra.run.trial127.stdout:(73/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 27 MB/s | 2.0 MB 00:00 2026-02-19T16:18:27.226 INFO:teuthology.orchestra.run.trial127.stdout:(74/96): python3-isodate-0.6.1-3.el9.noarch.rpm 2.0 MB/s | 56 kB 00:00 2026-02-19T16:18:27.230 INFO:teuthology.orchestra.run.trial127.stdout:(75/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 480 kB/s | 11 kB 00:00 2026-02-19T16:18:27.244 INFO:teuthology.orchestra.run.trial127.stdout:(76/96): python3-jaraco-classes-3.2.1-5.el9.noa 780 kB/s | 18 kB 00:00 2026-02-19T16:18:27.253 INFO:teuthology.orchestra.run.trial127.stdout:(77/96): python3-jaraco-collections-3.0.0-8.el9 869 kB/s | 23 kB 00:00 2026-02-19T16:18:27.253 INFO:teuthology.orchestra.run.trial127.stdout:(78/96): python3-jaraco-context-6.0.1-3.el9.noa 852 kB/s | 20 kB 00:00 2026-02-19T16:18:27.267 INFO:teuthology.orchestra.run.trial127.stdout:(79/96): python3-jaraco-functools-3.5.0-2.el9.n 859 kB/s | 19 kB 00:00 2026-02-19T16:18:27.280 INFO:teuthology.orchestra.run.trial127.stdout:(80/96): python3-jaraco-text-4.0.0-2.el9.noarch 971 kB/s | 26 kB 00:00 2026-02-19T16:18:27.290 INFO:teuthology.orchestra.run.trial127.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 3.3 MB/s | 79 kB 00:00 2026-02-19T16:18:27.309 INFO:teuthology.orchestra.run.trial127.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.9 MB/s | 86 kB 00:00 2026-02-19T16:18:27.313 INFO:teuthology.orchestra.run.trial127.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 2.5 MB/s | 58 kB 00:00 2026-02-19T16:18:27.317 INFO:teuthology.orchestra.run.trial127.stdout:(84/96): python3-kubernetes-26.1.0-3.el9.noarch 16 MB/s | 1.0 MB 00:00 2026-02-19T16:18:27.336 INFO:teuthology.orchestra.run.trial127.stdout:(85/96): python3-portend-3.1.0-2.el9.noarch.rpm 602 kB/s | 16 kB 00:00 2026-02-19T16:18:27.337 INFO:teuthology.orchestra.run.trial127.stdout:(86/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.7 MB/s | 90 kB 00:00 2026-02-19T16:18:27.339 INFO:teuthology.orchestra.run.trial127.stdout:(87/96): python3-repoze-lru-0.7-16.el9.noarch.r 1.3 MB/s | 31 kB 00:00 2026-02-19T16:18:27.360 INFO:teuthology.orchestra.run.trial127.stdout:(88/96): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2026-02-19T16:18:27.363 INFO:teuthology.orchestra.run.trial127.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 5.1 MB/s | 125 kB 00:00 2026-02-19T16:18:27.369 INFO:teuthology.orchestra.run.trial127.stdout:(90/96): python3-routes-2.5.1-5.el9.noarch.rpm 5.7 MB/s | 188 kB 00:00 2026-02-19T16:18:27.383 INFO:teuthology.orchestra.run.trial127.stdout:(91/96): python3-tempora-5.0.0-2.el9.noarch.rpm 1.5 MB/s | 36 kB 00:00 2026-02-19T16:18:27.387 INFO:teuthology.orchestra.run.trial127.stdout:(92/96): python3-typing-extensions-4.15.0-1.el9 3.6 MB/s | 86 kB 00:00 2026-02-19T16:18:27.398 INFO:teuthology.orchestra.run.trial127.stdout:(93/96): python3-websocket-client-1.2.3-2.el9.n 3.0 MB/s | 90 kB 00:00 2026-02-19T16:18:27.406 INFO:teuthology.orchestra.run.trial127.stdout:(94/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 2.1 MB/s | 48 kB 00:00 2026-02-19T16:18:27.409 INFO:teuthology.orchestra.run.trial127.stdout:(95/96): python3-xmltodict-0.12.0-15.el9.noarch 992 kB/s | 22 kB 00:00 2026-02-19T16:18:27.425 INFO:teuthology.orchestra.run.trial127.stdout:(96/96): python3-zc-lockfile-2.0-10.el9.noarch. 736 kB/s | 20 kB 00:00 2026-02-19T16:18:27.429 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:27.429 INFO:teuthology.orchestra.run.trial127.stdout:Total 7.2 MB/s | 159 MB 00:22 2026-02-19T16:18:27.860 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:27.899 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:27.899 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:28.503 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:28.503 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:29.148 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:29.149 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-19T16:18:29.154 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-19T16:18:29.175 INFO:teuthology.orchestra.run.trial127.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-19T16:18:29.209 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-19T16:18:29.215 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-19T16:18:29.218 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-19T16:18:29.222 INFO:teuthology.orchestra.run.trial127.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-19T16:18:29.245 INFO:teuthology.orchestra.run.trial127.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-19T16:18:29.258 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-19T16:18:29.261 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-19T16:18:29.267 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-19T16:18:29.269 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-19T16:18:29.280 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-19T16:18:29.286 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-19T16:18:29.310 INFO:teuthology.orchestra.run.trial127.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-19T16:18:29.314 INFO:teuthology.orchestra.run.trial127.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-19T16:18:29.364 INFO:teuthology.orchestra.run.trial127.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-19T16:18:29.377 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-19T16:18:29.379 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-19T16:18:29.384 INFO:teuthology.orchestra.run.trial127.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-19T16:18:29.389 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-19T16:18:29.394 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-19T16:18:29.397 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-19T16:18:29.411 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-19T16:18:29.416 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-19T16:18:29.421 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-19T16:18:29.432 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-19T16:18:29.441 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-19T16:18:29.449 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-19T16:18:29.469 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-19T16:18:29.481 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-19T16:18:29.496 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-19T16:18:29.501 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-19T16:18:29.508 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-19T16:18:29.548 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-19T16:18:29.877 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-19T16:18:29.887 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-19T16:18:29.891 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-19T16:18:29.897 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-19T16:18:29.899 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-19T16:18:29.926 INFO:teuthology.orchestra.run.trial127.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-19T16:18:29.970 INFO:teuthology.orchestra.run.trial127.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-19T16:18:29.978 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-19T16:18:29.983 INFO:teuthology.orchestra.run.trial127.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-19T16:18:29.987 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-19T16:18:29.993 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-19T16:18:29.996 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-19T16:18:30.003 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-19T16:18:30.007 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-19T16:18:30.035 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-19T16:18:30.045 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-19T16:18:30.050 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-19T16:18:30.057 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-19T16:18:30.092 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-19T16:18:30.316 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-19T16:18:30.340 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-19T16:18:30.344 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-19T16:18:30.402 INFO:teuthology.orchestra.run.trial127.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-19T16:18:30.404 INFO:teuthology.orchestra.run.trial127.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-19T16:18:30.425 INFO:teuthology.orchestra.run.trial127.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-19T16:18:30.755 INFO:teuthology.orchestra.run.trial127.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-19T16:18:30.829 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-19T16:18:31.504 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-19T16:18:31.521 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-19T16:18:31.531 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-19T16:18:31.538 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-19T16:18:31.551 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-19T16:18:31.567 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-19T16:18:31.643 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-19T16:18:31.652 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-19T16:18:31.675 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-19T16:18:31.705 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-19T16:18:31.756 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-19T16:18:31.762 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-19T16:18:31.765 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-19T16:18:31.767 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-19T16:18:31.786 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T16:18:31.786 INFO:teuthology.orchestra.run.trial127.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-19T16:18:31.786 INFO:teuthology.orchestra.run.trial127.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-19T16:18:31.786 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:31.793 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T16:18:31.819 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T16:18:31.819 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-19T16:18:31.819 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:31.853 INFO:teuthology.orchestra.run.trial127.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-19T16:18:32.477 INFO:teuthology.orchestra.run.trial127.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-19T16:18:33.251 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 80/96 2026-02-19T16:18:33.316 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 81/96 2026-02-19T16:18:33.319 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 82/96 2026-02-19T16:18:33.333 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 82/96 2026-02-19T16:18:33.333 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:18:33.333 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-19T16:18:33.334 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:33.345 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 83/96 2026-02-19T16:18:33.370 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 83/96 2026-02-19T16:18:33.370 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:18:33.370 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-19T16:18:33.370 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-19T16:18:33.370 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-19T16:18:33.370 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:33.432 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: cephadm-2:20.3.0-5336.g802bf111.el9.noarch 84/96 2026-02-19T16:18:33.434 INFO:teuthology.orchestra.run.trial127.stdout: Installing : cephadm-2:20.3.0-5336.g802bf111.el9.noarch 84/96 2026-02-19T16:18:33.443 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9 85/96 2026-02-19T16:18:33.467 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el 86/96 2026-02-19T16:18:33.468 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 87/96 2026-02-19T16:18:36.377 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 87/96 2026-02-19T16:18:36.384 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 88/96 2026-02-19T16:18:36.846 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 88/96 2026-02-19T16:18:36.847 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 89/96 2026-02-19T16:18:36.861 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 89/96 2026-02-19T16:18:36.861 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 90/96 2026-02-19T16:18:36.925 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 90/96 2026-02-19T16:18:36.967 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9. 91/96 2026-02-19T16:18:36.968 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 92/96 2026-02-19T16:18:36.989 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 92/96 2026-02-19T16:18:36.990 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:18:36.990 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-19T16:18:36.990 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-19T16:18:36.990 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-19T16:18:36.990 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:37.001 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 93/96 2026-02-19T16:18:37.223 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 93/96 2026-02-19T16:18:37.225 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 94/96 2026-02-19T16:18:37.247 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 94/96 2026-02-19T16:18:37.247 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:18:37.247 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-19T16:18:37.247 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-19T16:18:37.247 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-19T16:18:37.247 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:37.345 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 95/96 2026-02-19T16:18:37.366 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 95/96 2026-02-19T16:18:37.366 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:18:37.366 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-19T16:18:37.366 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-19T16:18:37.366 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-19T16:18:37.366 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:37.367 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-2:20.3.0-5336.g802bf111.el9.x86_64 96/96 2026-02-19T16:18:37.374 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 96/96 2026-02-19T16:18:37.398 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 96/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-2:20.3.0-5336.g802bf111.el9.x86_64 96/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-2:20.3.0-5336.g802bf111.el9.x86_64 1/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 2/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 3/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 4/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 5/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 6/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 7/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el 8/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 9/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 10/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 11/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 12/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9. 13/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 14/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9 15/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 16/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : cephadm-2:20.3.0-5336.g802bf111.el9.noarch 17/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-19T16:18:38.200 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-19T16:18:38.201 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-19T16:18:38.202 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-19T16:18:38.269 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-19T16:18:38.269 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:38.269 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:38.269 INFO:teuthology.orchestra.run.trial127.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-19T16:18:38.269 INFO:teuthology.orchestra.run.trial127.stdout: ceph-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:38.269 INFO:teuthology.orchestra.run.trial127.stdout: ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.269 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:38.270 INFO:teuthology.orchestra.run.trial127.stdout: ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: cephadm-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-19T16:18:38.271 INFO:teuthology.orchestra.run.trial127.stdout: lua-5.4.4-4.el9.x86_64 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-19T16:18:38.272 INFO:teuthology.orchestra.run.trial127.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-19T16:18:38.273 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-19T16:18:38.274 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-ply-3.11-14.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-19T16:18:38.275 INFO:teuthology.orchestra.run.trial127.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-19T16:18:38.276 INFO:teuthology.orchestra.run.trial127.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-19T16:18:38.277 INFO:teuthology.orchestra.run.trial127.stdout: unzip-6.0-59.el9.x86_64 2026-02-19T16:18:38.277 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-19T16:18:38.277 INFO:teuthology.orchestra.run.trial127.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-19T16:18:38.277 INFO:teuthology.orchestra.run.trial127.stdout: zip-3.0-35.el9.x86_64 2026-02-19T16:18:38.277 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:38.277 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:38.335 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-base 2026-02-19T16:18:38.652 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:29 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:38.718 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:38.736 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:38.737 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:38.737 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:38.759 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install cephadm 2026-02-19T16:18:39.072 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:30 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:39.137 INFO:teuthology.orchestra.run.trial127.stdout:Package cephadm-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T16:18:39.156 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:39.156 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:39.157 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:39.177 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-immutable-object-cache 2026-02-19T16:18:39.490 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:30 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5336.g802bf111.el9 ceph 147 k 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-19T16:18:39.574 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:39.575 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 147 k 2026-02-19T16:18:39.575 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 423 k 2026-02-19T16:18:39.575 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:39.968 INFO:teuthology.orchestra.run.trial127.stdout:ceph-immutable-object-cache-20.3.0-5336.g802bf1 375 kB/s | 147 kB 00:00 2026-02-19T16:18:39.969 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:39.969 INFO:teuthology.orchestra.run.trial127.stdout:Total 374 kB/s | 147 kB 00:00 2026-02-19T16:18:39.969 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:39.974 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:39.974 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:40.005 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:40.006 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:40.061 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:40.062 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T16:18:40.085 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T16:18:40.085 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:18:40.085 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-19T16:18:40.086 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:40.371 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T16:18:40.371 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:40.371 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:40.372 INFO:teuthology.orchestra.run.trial127.stdout: ceph-immutable-object-cache-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:40.372 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:40.372 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:40.398 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr 2026-02-19T16:18:40.714 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:31 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:40.778 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:40.796 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:40.797 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:40.797 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:40.818 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-dashboard 2026-02-19T16:18:41.133 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:32 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:41.200 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T16:18:41.218 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:41.219 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:41.219 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:41.241 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-19T16:18:41.554 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:32 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:41.620 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T16:18:41.638 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:41.639 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:41.639 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:41.660 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-rook 2026-02-19T16:18:41.971 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:32 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:42.036 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T16:18:42.054 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:42.055 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:42.055 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:42.076 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-mgr-cephadm 2026-02-19T16:18:42.391 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:33 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:42.458 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T16:18:42.476 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:42.476 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:42.477 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:42.496 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd 2026-02-19T16:18:42.807 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:33 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:42.873 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:42.891 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:42.892 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:42.892 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:42.912 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-osd-classic 2026-02-19T16:18:43.221 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:34 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:43.286 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:43.305 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:43.305 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:43.305 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:43.328 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-fuse 2026-02-19T16:18:43.647 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:34 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:43.733 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:43.734 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:43.734 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-19T16:18:43.734 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:43.734 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout: ceph-fuse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 987 k 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 987 k 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 2.9 M 2026-02-19T16:18:43.735 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:44.243 INFO:teuthology.orchestra.run.trial127.stdout:ceph-fuse-20.3.0-5336.g802bf111.el9.x86_64.rpm 1.9 MB/s | 987 kB 00:00 2026-02-19T16:18:44.243 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:44.243 INFO:teuthology.orchestra.run.trial127.stdout:Total 1.9 MB/s | 987 kB 00:00 2026-02-19T16:18:44.243 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:44.248 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:44.248 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:44.283 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:44.283 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:44.375 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:44.393 INFO:teuthology.orchestra.run.trial127.stdout: Installing : ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:44.632 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:44.690 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:44.690 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:44.690 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:44.690 INFO:teuthology.orchestra.run.trial127.stdout: ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:44.690 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:44.690 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:44.721 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install ceph-volume 2026-02-19T16:18:45.038 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:36 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:45.103 INFO:teuthology.orchestra.run.trial127.stdout:Package ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T16:18:45.122 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:45.123 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:45.123 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:45.144 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librados-devel 2026-02-19T16:18:45.461 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:36 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:45.545 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:45.546 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:45.546 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-19T16:18:45.546 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:45.546 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout: librados-devel x86_64 2:20.3.0-5336.g802bf111.el9 ceph 123 k 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 123 k 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 437 k 2026-02-19T16:18:45.547 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:45.926 INFO:teuthology.orchestra.run.trial127.stdout:librados-devel-20.3.0-5336.g802bf111.el9.x86_64 325 kB/s | 123 kB 00:00 2026-02-19T16:18:45.926 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:45.926 INFO:teuthology.orchestra.run.trial127.stdout:Total 324 kB/s | 123 kB 00:00 2026-02-19T16:18:45.926 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:45.929 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:45.929 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:45.960 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:45.960 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:46.014 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:46.030 INFO:teuthology.orchestra.run.trial127.stdout: Installing : librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:46.115 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:46.174 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:46.175 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:46.175 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:46.175 INFO:teuthology.orchestra.run.trial127.stdout: librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:46.175 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:46.175 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:46.203 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install libcephfs2 2026-02-19T16:18:46.516 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:37 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:46.580 INFO:teuthology.orchestra.run.trial127.stdout:Package libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:46.599 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:46.599 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:46.599 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:46.621 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install libcephfs-devel 2026-02-19T16:18:46.927 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:37 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:47.011 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:47.011 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:47.011 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-19T16:18:47.011 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-devel x86_64 2:20.3.0-5336.g802bf111.el9 ceph 37 k 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 37 k 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 171 k 2026-02-19T16:18:47.012 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:47.309 INFO:teuthology.orchestra.run.trial127.stdout:libcephfs-devel-20.3.0-5336.g802bf111.el9.x86_6 125 kB/s | 37 kB 00:00 2026-02-19T16:18:47.310 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:47.310 INFO:teuthology.orchestra.run.trial127.stdout:Total 125 kB/s | 37 kB 00:00 2026-02-19T16:18:47.310 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:47.312 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:47.312 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:47.315 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:47.315 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:47.336 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:47.353 INFO:teuthology.orchestra.run.trial127.stdout: Installing : libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:47.406 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:47.463 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:47.463 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:47.464 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:47.464 INFO:teuthology.orchestra.run.trial127.stdout: libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:47.464 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:47.464 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:47.490 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librados2 2026-02-19T16:18:47.801 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:38 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:47.865 INFO:teuthology.orchestra.run.trial127.stdout:Package librados2-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:47.884 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:47.884 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:47.885 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:47.906 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install librbd1 2026-02-19T16:18:48.221 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:39 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:48.288 INFO:teuthology.orchestra.run.trial127.stdout:Package librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:48.307 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:48.307 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:48.308 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:48.330 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rados 2026-02-19T16:18:48.644 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:39 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:48.708 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:48.727 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:48.727 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:48.727 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:48.749 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rgw 2026-02-19T16:18:49.063 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:40 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:49.128 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:49.146 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:49.147 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:49.147 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:49.169 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-cephfs 2026-02-19T16:18:49.476 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:40 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:49.542 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:49.560 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:49.561 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:49.561 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:49.582 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install python3-rbd 2026-02-19T16:18:49.893 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:40 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:49.958 INFO:teuthology.orchestra.run.trial127.stdout:Package python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T16:18:49.976 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:49.977 INFO:teuthology.orchestra.run.trial127.stdout:Nothing to do. 2026-02-19T16:18:49.977 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:49.998 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-fuse 2026-02-19T16:18:50.309 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:41 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:50.395 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout: rbd-fuse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 85 k 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:50.396 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-19T16:18:50.397 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:50.397 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 85 k 2026-02-19T16:18:50.397 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 217 k 2026-02-19T16:18:50.397 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:50.810 INFO:teuthology.orchestra.run.trial127.stdout:rbd-fuse-20.3.0-5336.g802bf111.el9.x86_64.rpm 204 kB/s | 85 kB 00:00 2026-02-19T16:18:50.811 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:50.811 INFO:teuthology.orchestra.run.trial127.stdout:Total 204 kB/s | 85 kB 00:00 2026-02-19T16:18:50.811 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:50.816 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:50.816 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:50.848 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:50.848 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:50.899 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:50.916 INFO:teuthology.orchestra.run.trial127.stdout: Installing : rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:50.997 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:51.054 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:51.055 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:51.055 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:51.055 INFO:teuthology.orchestra.run.trial127.stdout: rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:51.055 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:51.055 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:51.085 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-mirror 2026-02-19T16:18:51.396 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:42 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:51.480 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:51.480 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:51.480 INFO:teuthology.orchestra.run.trial127.stdout: Package Arch Version Repo Size 2026-02-19T16:18:51.480 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout: rbd-mirror x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.9 M 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 2.9 M 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 11 M 2026-02-19T16:18:51.481 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:52.540 INFO:teuthology.orchestra.run.trial127.stdout:rbd-mirror-20.3.0-5336.g802bf111.el9.x86_64.rpm 2.7 MB/s | 2.9 MB 00:01 2026-02-19T16:18:52.541 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:52.541 INFO:teuthology.orchestra.run.trial127.stdout:Total 2.7 MB/s | 2.9 MB 00:01 2026-02-19T16:18:52.541 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:52.546 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:52.546 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:52.585 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:52.585 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:52.763 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:52.764 INFO:teuthology.orchestra.run.trial127.stdout: Installing : rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:52.788 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:52.788 INFO:teuthology.orchestra.run.trial127.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T16:18:52.788 INFO:teuthology.orchestra.run.trial127.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-19T16:18:52.788 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-19T16:18:52.788 INFO:teuthology.orchestra.run.trial127.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-19T16:18:52.788 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:53.104 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:53.104 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:53.104 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:53.104 INFO:teuthology.orchestra.run.trial127.stdout: rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:53.105 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:53.105 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:53.135 DEBUG:teuthology.orchestra.run.trial127:> sudo yum -y install rbd-nbd 2026-02-19T16:18:53.449 INFO:teuthology.orchestra.run.trial127.stdout:Last metadata expiration check: 0:01:44 ago on Thu 19 Feb 2026 04:17:09 PM UTC. 2026-02-19T16:18:53.534 INFO:teuthology.orchestra.run.trial127.stdout:Dependencies resolved. 2026-02-19T16:18:53.534 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:53.534 INFO:teuthology.orchestra.run.trial127.stdout: Package Architecture Version Repository Size 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:Installing: 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout: rbd-nbd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 173 k 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:Transaction Summary 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:================================================================================ 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:Install 1 Package 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:Total download size: 173 k 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:Installed size: 466 k 2026-02-19T16:18:53.535 INFO:teuthology.orchestra.run.trial127.stdout:Downloading Packages: 2026-02-19T16:18:53.926 INFO:teuthology.orchestra.run.trial127.stdout:rbd-nbd-20.3.0-5336.g802bf111.el9.x86_64.rpm 443 kB/s | 173 kB 00:00 2026-02-19T16:18:53.928 INFO:teuthology.orchestra.run.trial127.stdout:-------------------------------------------------------------------------------- 2026-02-19T16:18:53.928 INFO:teuthology.orchestra.run.trial127.stdout:Total 441 kB/s | 173 kB 00:00 2026-02-19T16:18:53.928 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction check 2026-02-19T16:18:53.933 INFO:teuthology.orchestra.run.trial127.stdout:Transaction check succeeded. 2026-02-19T16:18:53.933 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction test 2026-02-19T16:18:53.966 INFO:teuthology.orchestra.run.trial127.stdout:Transaction test succeeded. 2026-02-19T16:18:53.966 INFO:teuthology.orchestra.run.trial127.stdout:Running transaction 2026-02-19T16:18:54.022 INFO:teuthology.orchestra.run.trial127.stdout: Preparing : 1/1 2026-02-19T16:18:54.039 INFO:teuthology.orchestra.run.trial127.stdout: Installing : rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:54.117 INFO:teuthology.orchestra.run.trial127.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:54.175 INFO:teuthology.orchestra.run.trial127.stdout: Verifying : rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T16:18:54.175 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:54.175 INFO:teuthology.orchestra.run.trial127.stdout:Installed: 2026-02-19T16:18:54.175 INFO:teuthology.orchestra.run.trial127.stdout: rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T16:18:54.175 INFO:teuthology.orchestra.run.trial127.stdout: 2026-02-19T16:18:54.175 INFO:teuthology.orchestra.run.trial127.stdout:Complete! 2026-02-19T16:18:54.208 DEBUG:teuthology.parallel:result is None 2026-02-19T16:18:54.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T16:18:54.377 DEBUG:teuthology.orchestra.run.trial127:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-19T16:18:54.398 INFO:teuthology.orchestra.run.trial127.stdout:20.3.0-5336.g802bf111.el9 2026-02-19T16:18:54.399 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5336.g802bf111.el9 2026-02-19T16:18:54.399 INFO:teuthology.task.install:The correct ceph version 20.3.0-5336.g802bf111 is installed. 2026-02-19T16:18:54.402 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-19T16:18:54.402 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-19T16:18:54.402 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-19T16:18:54.464 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-19T16:18:54.464 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-19T16:18:54.464 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/usr/bin/daemon-helper 2026-02-19T16:18:54.528 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-19T16:18:54.593 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-19T16:18:54.593 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-19T16:18:54.593 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-19T16:18:54.657 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-19T16:18:54.720 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-19T16:18:54.721 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-19T16:18:54.721 DEBUG:teuthology.orchestra.run.trial127:> sudo dd of=/usr/bin/stdin-killer 2026-02-19T16:18:54.784 DEBUG:teuthology.orchestra.run.trial127:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-19T16:18:54.850 INFO:teuthology.run_tasks:Running task workunit... 2026-02-19T16:18:54.873 INFO:tasks.workunit:Pulling workunits from ref 802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T16:18:54.873 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-19T16:18:54.873 INFO:tasks.workunit:timeout=3h 2026-02-19T16:18:54.873 INFO:tasks.workunit:cleanup=True 2026-02-19T16:18:54.874 DEBUG:teuthology.orchestra.run.trial127:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-19T16:18:54.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T16:18:54.907 INFO:teuthology.orchestra.run.trial127.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-19T16:18:54.907 DEBUG:teuthology.orchestra.run.trial127:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-19T16:18:54.963 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-19T16:18:54.963 DEBUG:teuthology.orchestra.run.trial127:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-19T16:18:55.019 DEBUG:teuthology.orchestra.run.trial127:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T16:18:55.082 INFO:tasks.workunit.client.0.trial127.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-19T16:19:14.555 INFO:tasks.workunit.client.0.trial127.stderr:Note: switching to '802bf111cd55e57f5842153d7ba2b8b964aa818a'. 2026-02-19T16:19:14.555 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-19T16:19:14.555 INFO:tasks.workunit.client.0.trial127.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-19T16:19:14.555 INFO:tasks.workunit.client.0.trial127.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-19T16:19:14.555 INFO:tasks.workunit.client.0.trial127.stderr:state without impacting any branches by switching back to a branch. 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: git switch -c 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr:Or undo this operation with: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: git switch - 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr: 2026-02-19T16:19:14.556 INFO:tasks.workunit.client.0.trial127.stderr:HEAD is now at 802bf111cd5 Merge branch 'wip-common-ceph_json-jfw-spirit-to-boost' of https://github.com/chardan/ceph into wip-yuri-testing-2026-02-18-1820 2026-02-19T16:19:14.562 DEBUG:teuthology.orchestra.run.trial127:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-19T16:19:14.619 DEBUG:teuthology.orchestra.run.trial127:> set -ex 2026-02-19T16:19:14.619 DEBUG:teuthology.orchestra.run.trial127:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-19T16:19:14.677 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-19T16:19:14.677 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-19T16:19:14.677 DEBUG:teuthology.orchestra.run.trial127:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=802bf111cd55e57f5842153d7ba2b8b964aa818a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:++ TIMEOUT=300 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:++ MAX_TIMEOUT=15 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:++ PG_NUM=4 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:++ TMPDIR=/tmp 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-19T16:19:14.739 INFO:tasks.workunit.client.0.trial127.stderr:+++ uname 2026-02-19T16:19:14.740 INFO:tasks.workunit.client.0.trial127.stderr:++ '[' Linux = FreeBSD ']' 2026-02-19T16:19:14.740 INFO:tasks.workunit.client.0.trial127.stderr:++ SED=sed 2026-02-19T16:19:14.740 INFO:tasks.workunit.client.0.trial127.stderr:++ AWK=awk 2026-02-19T16:19:14.740 INFO:tasks.workunit.client.0.trial127.stderr:+++ stty -a 2026-02-19T16:19:14.740 INFO:tasks.workunit.client.0.trial127.stderr:+++ head -1 2026-02-19T16:19:14.740 INFO:tasks.workunit.client.0.trial127.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-19T16:19:14.740 INFO:tasks.workunit.client.0.trial127.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T16:19:14.741 INFO:tasks.workunit.client.0.trial127.stderr:++ termwidth= 2026-02-19T16:19:14.741 INFO:tasks.workunit.client.0.trial127.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-19T16:19:14.741 INFO:tasks.workunit.client.0.trial127.stderr:++ DIFFCOLOPTS='-y ' 2026-02-19T16:19:14.741 INFO:tasks.workunit.client.0.trial127.stderr:++ KERNCORE=kernel.core_pattern 2026-02-19T16:19:14.741 INFO:tasks.workunit.client.0.trial127.stderr:++ EXTRA_OPTS= 2026-02-19T16:19:14.743 INFO:tasks.workunit.client.0.trial127.stderr:++ test '' = TESTS 2026-02-19T16:19:14.743 INFO:tasks.workunit.client.0.trial127.stderr:++ uname -m 2026-02-19T16:19:14.744 INFO:tasks.workunit.client.0.trial127.stderr:+ arch=x86_64 2026-02-19T16:19:14.744 INFO:tasks.workunit.client.0.trial127.stderr:+ case $arch in 2026-02-19T16:19:14.744 INFO:tasks.workunit.client.0.trial127.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-19T16:19:14.744 INFO:tasks.workunit.client.0.trial127.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-19T16:19:14.744 INFO:tasks.workunit.client.0.trial127.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:+ main test-erasure-code-plugins 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:+ shift 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:+ shopt -s -o xtrace 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code-plugins 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-19T16:19:14.745 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-19T16:19:14.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none ' 2026-02-19T16:19:14.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:14.746 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-19T16:19:14.746 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_blaum_roth_warning 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:TEST_ec_profile_warning 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:TEST_preload_no_warning 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:TEST_preload_no_warning_default 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:TEST_preload_warning' 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_blaum_roth_warning td/test-erasure-code-plugins 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:119: TEST_ec_profile_blaum_roth_warning: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:121: TEST_ec_profile_blaum_roth_warning: setup td/test-erasure-code-plugins 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:14.748 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:14.749 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:14.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:14.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:14.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:14.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:14.750 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:14.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:14.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:14.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:14.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:14.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:14.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:14.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:14.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:14.753 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:14.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:14.755 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:14.755 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:14.756 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:14.756 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:14.756 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:14.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:14.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:14.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:14.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:19:14.758 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:19:14.758 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:14.758 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:14.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:19:14.759 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:122: TEST_ec_profile_blaum_roth_warning: run_mon td/test-erasure-code-plugins a 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:19:14.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.787 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:19:14.788 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:14.788 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:14.788 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:14.788 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:14.788 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:14.788 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:14.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:19:14.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:19:14.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:19:14.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:14.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:14.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:14.813 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:19:14.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:19:14.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:14.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:14.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:19:14.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:123: TEST_ec_profile_blaum_roth_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:19:14.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:14.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:19:14.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:19:14.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:19:14.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:19:14.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:19:14.994 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:19:14.994 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:14.994 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:14.994 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:14.995 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:14.995 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:14.995 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:14.995 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:14.995 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:15.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: seq 0 2 2026-02-19T16:19:15.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-19T16:19:15.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 0 2026-02-19T16:19:15.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:15.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:15.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:19:15.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:15.014 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d2312a7d-e3f0-45d8-b269-78af288b6728 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d2312a7d-e3f0-45d8-b269-78af288b6728' 2026-02-19T16:19:15.016 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 d2312a7d-e3f0-45d8-b269-78af288b6728 2026-02-19T16:19:15.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:15.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQADOJdpyaQ7ARAA3FiG4s2ttaLAaa2y6QsfEw== 2026-02-19T16:19:15.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQADOJdpyaQ7ARAA3FiG4s2ttaLAaa2y6QsfEw=="}' 2026-02-19T16:19:15.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d2312a7d-e3f0-45d8-b269-78af288b6728 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:15.122 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:15.128 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:15.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQADOJdpyaQ7ARAA3FiG4s2ttaLAaa2y6QsfEw== --osd-uuid d2312a7d-e3f0-45d8-b269-78af288b6728 2026-02-19T16:19:15.143 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.141+0000 7f745e63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:15.145 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.143+0000 7f745e63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:15.145 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.144+0000 7f745e63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:15.146 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.144+0000 7f745e63a780 -1 bdev(0x31dec00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:15.146 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.144+0000 7f745e63a780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:19:15.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:19:15.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:15.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:19:15.517 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:19:15.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:15.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:19:15.620 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:19:15.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:19:15.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:15.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:15.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:15.633 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.631+0000 7f578e262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:15.635 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.633+0000 7f578e262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:15.636 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.634+0000 7f578e262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:15.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:19:15.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:15.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:19:15.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:15.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:15.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:15.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:15.718 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:15.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:15.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:15.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:15.820 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:15.818+0000 7f578e262780 -1 Falling back to public interface 2026-02-19T16:19:16.142 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:16.140+0000 7f578e262780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:19:16.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:16.816 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:16.912 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3368676727,v1:127.0.0.1:6801/3368676727] [v2:127.0.0.1:6802/3368676727,v1:127.0.0.1:6803/3368676727] exists,up d2312a7d-e3f0-45d8-b269-78af288b6728 2026-02-19T16:19:16.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:16.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 1 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:16.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:16.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:19:16.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-19T16:19:16.920 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:16.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5b7b84b9-d488-4d67-9b0e-e0a3e4c57e62 2026-02-19T16:19:16.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5b7b84b9-d488-4d67-9b0e-e0a3e4c57e62' 2026-02-19T16:19:16.920 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 5b7b84b9-d488-4d67-9b0e-e0a3e4c57e62 2026-02-19T16:19:16.920 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:16.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAEOJdpLmYINxAATgZFeb9YlfSUn+zUmTL/RA== 2026-02-19T16:19:16.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAEOJdpLmYINxAATgZFeb9YlfSUn+zUmTL/RA=="}' 2026-02-19T16:19:16.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5b7b84b9-d488-4d67-9b0e-e0a3e4c57e62 -i td/test-erasure-code-plugins/1/new.json 2026-02-19T16:19:17.024 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:17.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-19T16:19:17.030 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAEOJdpLmYINxAATgZFeb9YlfSUn+zUmTL/RA== --osd-uuid 5b7b84b9-d488-4d67-9b0e-e0a3e4c57e62 2026-02-19T16:19:17.045 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.044+0000 7f241f064780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:17.047 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.045+0000 7f241f064780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:17.048 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.046+0000 7f241f064780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:17.048 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.046+0000 7f241f064780 -1 bdev(0x4062c00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:17.048 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.047+0000 7f241f064780 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-19T16:19:17.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-19T16:19:17.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:17.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:19:17.422 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:19:17.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:17.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:19:17.732 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:19:17.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:19:17.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:17.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:17.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:17.745 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.744+0000 7f8ff500c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:17.747 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.745+0000 7f8ff500c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:17.748 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.746+0000 7f8ff500c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:17.869 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:17.867+0000 7f8ff500c780 -1 Falling back to public interface 2026-02-19T16:19:17.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:19:17.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:17.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:19:17.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:17.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:17.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:17.962 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:17.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:17.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:17.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:19:18.140 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:18.138+0000 7f8ff500c780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:19:18.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:19.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:19.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:19.189 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:19.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:19.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:19.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:19:19.420 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1624278717,v1:127.0.0.1:6811/1624278717] [v2:127.0.0.1:6812/1624278717,v1:127.0.0.1:6813/1624278717] exists,up 5b7b84b9-d488-4d67-9b0e-e0a3e4c57e62 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 2 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:19:19.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:19.425 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:19.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0870b293-cbc8-4111-9d9c-cfe888ffbe6d 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 0870b293-cbc8-4111-9d9c-cfe888ffbe6d' 2026-02-19T16:19:19.427 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 0870b293-cbc8-4111-9d9c-cfe888ffbe6d 2026-02-19T16:19:19.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:19.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAHOJdpnzCyGRAAnWVIxDCquFOITJfyj84RSg== 2026-02-19T16:19:19.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAHOJdpnzCyGRAAnWVIxDCquFOITJfyj84RSg=="}' 2026-02-19T16:19:19.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0870b293-cbc8-4111-9d9c-cfe888ffbe6d -i td/test-erasure-code-plugins/2/new.json 2026-02-19T16:19:19.662 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:19:19.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-19T16:19:19.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAHOJdpnzCyGRAAnWVIxDCquFOITJfyj84RSg== --osd-uuid 0870b293-cbc8-4111-9d9c-cfe888ffbe6d 2026-02-19T16:19:19.686 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:19.684+0000 7f7e62810780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:19.687 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:19.685+0000 7f7e62810780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:19.688 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:19.686+0000 7f7e62810780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:19.689 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:19.687+0000 7f7e62810780 -1 bdev(0x2482c00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:19.689 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:19.687+0000 7f7e62810780 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-19T16:19:20.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-19T16:19:20.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:20.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:19:20.004 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:19:20.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:20.322 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:19:20.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:19:20.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:20.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:20.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:20.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:19:20.336 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:20.335+0000 7fec5fe33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:20.338 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:20.336+0000 7fec5fe33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:20.339 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:20.337+0000 7fec5fe33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:20.510 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:20.508+0000 7fec5fe33780 -1 Falling back to public interface 2026-02-19T16:19:20.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:19:20.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:20.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:19:20.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:20.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:20.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:20.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:20.555 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:20.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:20.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:19:20.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:20.816 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:20.814+0000 7fec5fe33780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:19:21.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:21.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:21.778 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:21.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:21.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:21.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:19:22.004 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2766828800,v1:127.0.0.1:6819/2766828800] [v2:127.0.0.1:6820/2766828800,v1:127.0.0.1:6821/2766828800] exists,up 0870b293-cbc8-4111-9d9c-cfe888ffbe6d 2026-02-19T16:19:22.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:22.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:22.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:22.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:127: TEST_ec_profile_blaum_roth_warning: create_rbd_pool 2026-02-19T16:19:22.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T16:19:22.225 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' does not exist 2026-02-19T16:19:22.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T16:19:22.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:19:22.479 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:19:22.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:19:23.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T16:19:23.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:128: TEST_ec_profile_blaum_roth_warning: wait_for_clean 2026-02-19T16:19:23.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:19:23.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:19:23.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:19:23.771 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:19:23.771 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:19:23.771 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:19:23.771 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:19:23.771 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:19:23.771 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:19:23.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:19:23.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:19:23.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:19:23.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:19:23.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:19:23.853 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:19:24.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:19:24.085 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:19:24.085 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:19:24.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:19:24.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:19:24.085 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:19:24.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T16:19:24.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T16:19:24.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T16:19:24.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:19:24.149 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:19:24.214 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T16:19:24.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T16:19:24.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-19T16:19:24.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:19:24.215 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:19:24.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-19T16:19:24.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-19T16:19:24.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-55834574851' 2026-02-19T16:19:24.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:19:24.278 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-19T16:19:24.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:19:24.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:19:24.280 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T16:19:24.280 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:19:24.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-19T16:19:24.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T16:19:24.281 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 21474836484 2026-02-19T16:19:24.282 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:19:24.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-19T16:19:24.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:19:25.508 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:19:25.509 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:19:25.739 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-19T16:19:25.739 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:19:25.739 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705667 2026-02-19T16:19:25.739 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:19:25.740 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:19:25.741 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T16:19:25.741 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:19:25.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705667 2026-02-19T16:19:25.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T16:19:25.742 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 38654705667 2026-02-19T16:19:25.742 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:19:25.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-19T16:19:25.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:19:25.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574851 2026-02-19T16:19:25.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:19:25.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:19:25.970 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-19T16:19:25.970 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:19:25.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574851 2026-02-19T16:19:25.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-19T16:19:25.972 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 55834574851 2026-02-19T16:19:25.972 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:19:26.196 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-19T16:19:26.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:19:26.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:19:26.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:19:26.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:19:26.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T16:19:26.737 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:19:26.737 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:19:26.737 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:19:27.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T16:19:27.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:19:27.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:19:27.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:130: TEST_ec_profile_blaum_roth_warning: echo 'Starting test for blaum-roth profile health warning' 2026-02-19T16:19:27.050 INFO:tasks.workunit.client.0.trial127.stdout:Starting test for blaum-roth profile health warning 2026-02-19T16:19:27.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:133: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set prof- plugin=jerasure k=3 m=1 technique=blaum_roth w=7 --yes-i-really-mean-it --force 2026-02-19T16:19:27.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:135: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set test crush-device-class= crush-failure-domain=osd crush-num-failure-domains=0 crush-osds-per-failure-domain=0 crush-root=default k=4 l=3 m=2 name=lrcprofile plugin=lrc 2026-02-19T16:19:27.633 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: get_asok_path mon.a 2026-02-19T16:19:27.633 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:27.633 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:27.633 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:27.633 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:27.633 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:27.633 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:27.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: CEPH_ARGS= 2026-02-19T16:19:27.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:27.672 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:27.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:137: TEST_ec_profile_blaum_roth_warning: sleep 10 2026-02-19T16:19:37.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:138: TEST_ec_profile_blaum_roth_warning: grep -F '1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:29.810+0000 7f787bbf6640 0 log_channel(cluster) log [WRN] : Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:29.811+0000 7f78793f1640 10 mon.a@0(leader).log v75 logging 2026-02-19T16:19:29.811180+0000 mon.a (mon.0) 141 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:29.910+0000 7f7876bec640 7 mon.a@0(leader).log v76 update_from_paxos applying incremental log 76 2026-02-19T16:19:29.811180+0000 mon.a (mon.0) 141 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.681 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.682 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.682 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.682 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.682 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.682 INFO:tasks.workunit.client.0.trial127.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-19T16:19:37.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:139: TEST_ec_profile_blaum_roth_warning: grep -F 'w+1=8 for the EC profile prof- is not prime and could lead to data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:37.683 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.683 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.683 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.684 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.684 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.684 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.684 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.684 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.684 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.684 INFO:tasks.workunit.client.0.trial127.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-19T16:19:37.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:141: TEST_ec_profile_blaum_roth_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:19:37.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:37.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:37.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:37.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:37.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:37.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:37.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:37.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:37.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:37.793 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:37.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:37.794 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:37.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:37.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:37.795 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:37.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:37.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:37.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:37.797 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:37.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:37.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:37.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:37.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:37.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:37.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:37.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:142: TEST_ec_profile_blaum_roth_warning: return 0 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-19T16:19:37.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:37.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:37.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:37.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:37.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:37.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:37.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:37.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:37.811 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:37.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:37.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:37.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:37.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:37.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:37.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:37.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:37.815 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:37.816 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:37.816 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:37.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:37.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:37.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:37.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:19:37.818 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:19:37.818 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:37.818 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:37.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:19:37.819 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:19:37.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:19:37.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-19T16:19:37.838 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:19:37.838 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:37.838 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:37.838 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:37.838 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:37.838 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:37.838 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:37.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:19:37.861 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:19:37.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:19:37.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:37.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:37.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:19:37.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:19:37.863 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:37.863 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:37.864 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:37.864 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:37.864 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:37.864 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:37.864 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:37.864 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:37.865 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:37.907 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:37.908 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:37.908 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:37.908 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:37.908 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:37.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:37.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:19:37.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:19:37.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:37.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:19:37.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:19:37.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:19:37.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:19:37.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:19:38.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:19:38.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:38.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:38.046 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:38.046 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:38.046 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:38.046 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:38.046 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:38.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:38.061 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:38.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:38.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:19:38.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:19:38.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:38.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=22dadaaa-52c9-4851-9db7-40c9ce690ee8 2026-02-19T16:19:38.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 22dadaaa-52c9-4851-9db7-40c9ce690ee8' 2026-02-19T16:19:38.064 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 22dadaaa-52c9-4851-9db7-40c9ce690ee8 2026-02-19T16:19:38.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:38.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAaOJdpk6VgBBAAyarcg1iVF9L6t5y+KUU6pw== 2026-02-19T16:19:38.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAaOJdpk6VgBBAAyarcg1iVF9L6t5y+KUU6pw=="}' 2026-02-19T16:19:38.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 22dadaaa-52c9-4851-9db7-40c9ce690ee8 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:38.176 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:38.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:38.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAaOJdpk6VgBBAAyarcg1iVF9L6t5y+KUU6pw== --osd-uuid 22dadaaa-52c9-4851-9db7-40c9ce690ee8 2026-02-19T16:19:38.198 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.195+0000 7f6d7140f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:38.200 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.197+0000 7f6d7140f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:38.200 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.198+0000 7f6d7140f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:38.201 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.198+0000 7f6d7140f780 -1 bdev(0x31b4c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:38.201 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.198+0000 7f6d7140f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:19:38.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:19:38.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:38.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:19:38.589 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:19:38.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:38.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:19:38.691 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:19:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:19:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:38.704 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.701+0000 7f1590042780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:38.705 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.703+0000 7f1590042780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:38.706 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.703+0000 7f1590042780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:38.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:19:38.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:38.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:19:38.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:38.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:38.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:38.788 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:38.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:38.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:38.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:38.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:38.910 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:38.907+0000 7f1590042780 -1 Falling back to public interface 2026-02-19T16:19:39.288 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:39.285+0000 7f1590042780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:19:39.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:39.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:39.884 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:39.885 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:39.885 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:39.885 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:39.981 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1925730832,v1:127.0.0.1:6801/1925730832] [v2:127.0.0.1:6802/1925730832,v1:127.0.0.1:6803/1925730832] exists,up 22dadaaa-52c9-4851-9db7-40c9ce690ee8 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:39.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:39.986 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:39.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0cdbc857-2519-4b8c-9bfa-637778503be2 2026-02-19T16:19:39.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 0cdbc857-2519-4b8c-9bfa-637778503be2' 2026-02-19T16:19:39.989 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 0cdbc857-2519-4b8c-9bfa-637778503be2 2026-02-19T16:19:39.989 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:39.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAbOJdpBNMeOxAAGxuNbQk18sqgB3HcrmBxwg== 2026-02-19T16:19:39.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAbOJdpBNMeOxAAGxuNbQk18sqgB3HcrmBxwg=="}' 2026-02-19T16:19:39.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0cdbc857-2519-4b8c-9bfa-637778503be2 -i td/test-erasure-code-plugins/1/new.json 2026-02-19T16:19:40.097 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:40.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-19T16:19:40.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAbOJdpBNMeOxAAGxuNbQk18sqgB3HcrmBxwg== --osd-uuid 0cdbc857-2519-4b8c-9bfa-637778503be2 2026-02-19T16:19:40.120 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.117+0000 7f9217629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:40.122 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.119+0000 7f9217629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:40.123 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.120+0000 7f9217629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:40.123 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.120+0000 7f9217629780 -1 bdev(0x40e4c00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:40.123 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.120+0000 7f9217629780 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-19T16:19:40.451 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-19T16:19:40.451 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:40.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:19:40.452 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:19:40.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:40.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:19:40.770 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:19:40.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:19:40.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:40.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:40.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:40.783 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.780+0000 7f786e7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:40.785 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.782+0000 7f786e7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:40.786 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.783+0000 7f786e7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:40.917 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:40.914+0000 7f786e7d8780 -1 Falling back to public interface 2026-02-19T16:19:40.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:19:40.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:40.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:19:40.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:40.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:40.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:40.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:40.995 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:40.995 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:40.995 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:19:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:41.247 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:41.245+0000 7f786e7d8780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:19:42.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:42.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:42.225 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:42.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:42.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:42.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:19:42.449 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3416904164,v1:127.0.0.1:6811/3416904164] [v2:127.0.0.1:6812/3416904164,v1:127.0.0.1:6813/3416904164] exists,up 0cdbc857-2519-4b8c-9bfa-637778503be2 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:42.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-19T16:19:42.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:42.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f817e610-a1a2-40f0-aef5-6ef76db2cf18 2026-02-19T16:19:42.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f817e610-a1a2-40f0-aef5-6ef76db2cf18' 2026-02-19T16:19:42.456 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 f817e610-a1a2-40f0-aef5-6ef76db2cf18 2026-02-19T16:19:42.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:42.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAeOJdpNlJyGxAAX6HdJ0I+pxVf/EIeBAKdKQ== 2026-02-19T16:19:42.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAeOJdpNlJyGxAAX6HdJ0I+pxVf/EIeBAKdKQ=="}' 2026-02-19T16:19:42.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f817e610-a1a2-40f0-aef5-6ef76db2cf18 -i td/test-erasure-code-plugins/2/new.json 2026-02-19T16:19:42.695 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:19:42.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-19T16:19:42.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAeOJdpNlJyGxAAX6HdJ0I+pxVf/EIeBAKdKQ== --osd-uuid f817e610-a1a2-40f0-aef5-6ef76db2cf18 2026-02-19T16:19:42.719 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:42.716+0000 7f3d9484d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:42.720 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:42.717+0000 7f3d9484d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:42.721 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:42.718+0000 7f3d9484d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:42.721 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:42.719+0000 7f3d9484d780 -1 bdev(0x35ecc00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:42.721 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:42.719+0000 7f3d9484d780 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-19T16:19:43.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-19T16:19:43.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:43.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:19:43.043 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:19:43.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:43.363 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:19:43.364 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:19:43.364 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:19:43.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:43.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:43.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:43.377 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:43.374+0000 7fa2b6c36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:43.378 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:43.376+0000 7fa2b6c36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:43.379 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:43.377+0000 7fa2b6c36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:43.566 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:43.564+0000 7fa2b6c36780 -1 Falling back to public interface 2026-02-19T16:19:43.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:19:43.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:43.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:19:43.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:43.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:43.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:43.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:43.590 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:43.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:43.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:19:43.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:43.883 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:43.880+0000 7fa2b6c36780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:19:44.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:44.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:44.813 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:44.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:44.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:44.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:19:45.037 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1370322807,v1:127.0.0.1:6819/1370322807] [v2:127.0.0.1:6820/1370322807,v1:127.0.0.1:6821/1370322807] exists,up f817e610-a1a2-40f0-aef5-6ef76db2cf18 2026-02-19T16:19:45.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:45.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:45.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:45.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-19T16:19:45.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T16:19:45.257 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' does not exist 2026-02-19T16:19:45.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T16:19:45.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:19:45.537 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:19:45.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:19:46.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T16:19:46.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-19T16:19:46.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:19:46.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:19:46.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:19:46.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:19:46.829 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:19:46.829 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:19:46.830 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:19:46.830 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:19:46.830 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:19:46.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:19:46.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:19:46.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:19:46.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:19:46.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:19:46.912 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:19:47.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:19:47.137 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:19:47.137 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:19:47.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:19:47.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:19:47.137 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:19:47.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T16:19:47.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T16:19:47.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T16:19:47.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:19:47.203 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:19:47.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-19T16:19:47.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-19T16:19:47.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672963' 2026-02-19T16:19:47.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:19:47.266 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:19:47.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-19T16:19:47.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-19T16:19:47.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672963 2-60129542147' 2026-02-19T16:19:47.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:19:47.328 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-19T16:19:47.328 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:19:47.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:19:47.328 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T16:19:47.329 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:19:47.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-19T16:19:47.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T16:19:47.330 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 21474836484 2026-02-19T16:19:47.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:19:47.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-19T16:19:47.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:19:48.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:19:48.558 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:19:48.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-19T16:19:48.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:19:48.786 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672963 2026-02-19T16:19:48.786 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:19:48.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:19:48.787 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-19T16:19:48.788 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:19:48.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672963 2026-02-19T16:19:48.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-19T16:19:48.789 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 42949672963 2026-02-19T16:19:48.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:19:49.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672963 2026-02-19T16:19:49.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:19:49.018 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542147 2026-02-19T16:19:49.019 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:19:49.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:19:49.020 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-19T16:19:49.020 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:19:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542147 2026-02-19T16:19:49.021 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 60129542147 2026-02-19T16:19:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-19T16:19:49.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:19:49.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-19T16:19:49.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:19:49.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:19:49.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:19:49.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T16:19:49.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:19:49.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:19:49.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:19:49.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:19:49.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:19:49.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:19:49.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:19:49.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T16:19:49.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:19:49.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:19:49.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:19:50.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T16:19:50.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:19:50.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:19:50.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-19T16:19:50.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-19T16:19:50.384 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T16:19:50.384 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:50.384 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:50.384 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:50.385 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:50.385 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:50.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:50.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T16:19:50.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:50.423 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:50.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:50.431 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:50.255+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:50.431 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:50.373+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:50.431 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:50.373+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:50.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-19T16:19:50.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-19T16:19:50.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T16:19:50.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:50.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:50.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:50.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:50.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:50.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:50.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T16:19:50.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:50.790 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:50.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:50.795 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:50.622+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:50.796 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:50.740+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:50.796 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:50.740+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:50.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-19T16:19:50.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T16:19:51.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:51.119 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:51.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:51.124 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:50.953+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:51.124 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.070+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:51.124 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.070+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-19T16:19:51.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-19T16:19:51.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T16:19:51.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:51.449 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:51.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:51.455 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.281+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-19T16:19:51.455 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.399+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-19T16:19:51.455 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.399+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-19T16:19:51.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-19T16:19:51.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-19T16:19:51.741 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T16:19:51.741 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:51.741 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:51.741 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:51.741 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:51.741 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:51.741 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:51.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T16:19:51.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:51.779 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:51.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:51.785 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.613+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-19T16:19:51.785 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.731+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-19T16:19:51.785 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.731+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-19T16:19:51.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-19T16:19:51.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-19T16:19:52.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T16:19:52.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:52.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:52.073 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:52.073 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.073 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.074 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:52.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T16:19:52.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:52.113 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:52.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:52.119 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:51.943+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-19T16:19:52.119 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:52.062+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:19:52.062+0000 7f2e5c7f1640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:52.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:52.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:52.228 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:52.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:52.229 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:52.230 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:52.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:52.231 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:52.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:52.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:52.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:52.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:52.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:52.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:52.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:52.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:52.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:19:52.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:52.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:52.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:52.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:52.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:52.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:52.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:52.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:52.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:52.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:52.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:52.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:52.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:52.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:52.247 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:52.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:52.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:52.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:52.248 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:52.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:52.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:52.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:52.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:52.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:52.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:52.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:52.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:19:52.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:19:52.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.254 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:19:52.254 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:19:52.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:19:52.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:19:52.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:19:52.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-19T16:19:52.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:52.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:19:52.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:19:52.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:19:52.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:19:52.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:52.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-19T16:19:52.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:19:52.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:19:52.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:52.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:52.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:19:52.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:52.301 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:52.302 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:19:52.342 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:19:52.342 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:52.343 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:52.344 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:19:52.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:19:52.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:52.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:19:52.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:19:52.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:19:52.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:19:52.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:19:52.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:19:52.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:52.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:52.485 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:52.485 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.485 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:52.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:52.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:52.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T16:19:52.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:52.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:52.501 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:52.501 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.502 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:52.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:19:52.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:52.539 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:19:52.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:52.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:52.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:52.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:52.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-19T16:19:52.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:19:52.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:52.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a17b6648-7f26-4ac8-bcef-03ea90b494c2 2026-02-19T16:19:52.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a17b6648-7f26-4ac8-bcef-03ea90b494c2' 2026-02-19T16:19:52.548 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 a17b6648-7f26-4ac8-bcef-03ea90b494c2 2026-02-19T16:19:52.548 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:52.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAoOJdp6vL+IBAAH6qobneS71mdwksE92tPuQ== 2026-02-19T16:19:52.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAoOJdp6vL+IBAAH6qobneS71mdwksE92tPuQ=="}' 2026-02-19T16:19:52.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a17b6648-7f26-4ac8-bcef-03ea90b494c2 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:52.656 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:52.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:52.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQAoOJdp6vL+IBAAH6qobneS71mdwksE92tPuQ== --osd-uuid a17b6648-7f26-4ac8-bcef-03ea90b494c2 2026-02-19T16:19:52.677 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:52.675+0000 7f326f20c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:52.679 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:52.677+0000 7f326f20c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:52.679 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:52.678+0000 7f326f20c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:52.680 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:52.678+0000 7f326f20c780 -1 bdev(0x416ec00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:52.680 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:52.678+0000 7f326f20c780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:19:53.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:19:53.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:53.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:19:53.105 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:19:53.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:53.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:19:53.207 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:19:53.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-19T16:19:53.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:53.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:53.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:53.221 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:53.219+0000 7fbcd0e70780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:53.222 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:53.220+0000 7fbcd0e70780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:53.223 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:53.221+0000 7fbcd0e70780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:53.303 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:19:53.303 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:53.303 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:19:53.303 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:53.303 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:53.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:53.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:53.304 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:53.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:53.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:53.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:53.416 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:53.414+0000 7fbcd0e70780 -1 Falling back to public interface 2026-02-19T16:19:53.766 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:53.764+0000 7fbcd0e70780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:19:54.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:54.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:54.400 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:54.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:54.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:54.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:54.496 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3159326758,v1:127.0.0.1:6801/3159326758] [v2:127.0.0.1:6802/3159326758,v1:127.0.0.1:6803/3159326758] exists,up a17b6648-7f26-4ac8-bcef-03ea90b494c2 2026-02-19T16:19:54.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:54.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:54.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:54.498 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:19:54.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:19:54.543 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:54.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:54.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:19:54.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:54.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:54.656 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:54.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:54.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:54.658 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:54.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:54.660 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:54.660 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:54.660 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:54.660 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:54.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:54.661 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:54.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:54.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:54.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:54.666 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:54.666 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.666 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:54.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:54.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:54.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T16:19:54.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T16:19:54.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:54.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:54.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:54.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:54.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:54.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:54.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:54.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:54.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:54.671 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:54.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:54.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:54.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:54.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:54.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:54.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:54.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:54.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:54.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:54.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:54.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:54.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:19:54.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:19:54.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:19:54.679 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:19:54.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:19:54.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:19:54.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:19:54.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-19T16:19:54.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:54.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:19:54.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:19:54.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:19:54.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:19:54.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:54.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-19T16:19:54.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:19:54.722 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:19:54.722 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:54.722 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:54.723 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:19:54.723 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:19:54.724 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:54.724 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:54.724 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:54.724 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:54.724 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.724 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.724 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:54.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:54.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:19:54.768 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:19:54.768 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:54.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:54.770 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:19:54.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:19:54.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:54.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:19:54.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:19:54.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:19:54.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:19:54.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:19:54.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:19:54.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:54.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:54.909 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:54.909 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.909 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:54.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:54.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:19:54.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:54.965 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:54.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-19T16:19:54.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:19:54.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:54.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:54.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=634a74a6-7310-4f3c-9a86-f1f1d9e4dbb3 2026-02-19T16:19:54.976 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 634a74a6-7310-4f3c-9a86-f1f1d9e4dbb3 2026-02-19T16:19:54.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 634a74a6-7310-4f3c-9a86-f1f1d9e4dbb3' 2026-02-19T16:19:54.976 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:54.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAqOJdpbExdOhAAbfLNUFvQI7hIZKCqdcSFdA== 2026-02-19T16:19:54.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAqOJdpbExdOhAAbfLNUFvQI7hIZKCqdcSFdA=="}' 2026-02-19T16:19:54.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 634a74a6-7310-4f3c-9a86-f1f1d9e4dbb3 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:55.082 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:55.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:55.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQAqOJdpbExdOhAAbfLNUFvQI7hIZKCqdcSFdA== --osd-uuid 634a74a6-7310-4f3c-9a86-f1f1d9e4dbb3 2026-02-19T16:19:55.103 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.102+0000 7f13c6050780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:55.105 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.103+0000 7f13c6050780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:55.106 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.104+0000 7f13c6050780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:55.106 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.104+0000 7f13c6050780 -1 bdev(0x29e2c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:55.106 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.104+0000 7f13c6050780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:19:55.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:19:55.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:55.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:19:55.518 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:19:55.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:55.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:19:55.621 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:19:55.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-19T16:19:55.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:55.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:55.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:55.635 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.633+0000 7f7f30210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:55.636 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.634+0000 7f7f30210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:55.637 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.635+0000 7f7f30210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:55.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:19:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:19:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:55.717 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:55.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:55.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:55.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:55.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:55.845 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:55.843+0000 7f7f30210780 -1 Falling back to public interface 2026-02-19T16:19:56.164 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:56.162+0000 7f7f30210780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:19:56.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:56.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:56.814 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:56.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:56.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:56.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:56.911 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2120037307,v1:127.0.0.1:6801/2120037307] [v2:127.0.0.1:6802/2120037307,v1:127.0.0.1:6803/2120037307] exists,up 634a74a6-7310-4f3c-9a86-f1f1d9e4dbb3 2026-02-19T16:19:56.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:56.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:56.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:56.911 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T16:19:56.911 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:19:56.911 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:19:56.912 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:56.912 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:56.912 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:56.912 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:19:56.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:19:56.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:19:56.957 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:56.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:56.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:19:56.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:19:56.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:56.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:56.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:56.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:56.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:56.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:56.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:56.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:57.071 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:57.072 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:57.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:57.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:57.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:57.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:57.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:57.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:57.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:57.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:57.076 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:57.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:57.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:57.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:57.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:57.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:57.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:57.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:57.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:57.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:57.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:57.085 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:57.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:57.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:57.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:57.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:57.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:57.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:57.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:57.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:57.088 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:57.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:57.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:57.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:57.091 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:57.091 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.091 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:57.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:57.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:57.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:19:57.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:19:57.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.094 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:19:57.095 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:19:57.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:19:57.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:19:57.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:19:57.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-19T16:19:57.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:57.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:19:57.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:19:57.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:19:57.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:19:57.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-19T16:19:57.112 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:19:57.113 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:57.113 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:57.113 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:57.113 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.113 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.113 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:57.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-19T16:19:57.134 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:19:57.135 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:19:57.135 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:57.135 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:57.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:19:57.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:19:57.137 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:57.137 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:57.137 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:57.138 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:57.138 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.138 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.138 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:57.138 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:57.138 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:57.181 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:57.182 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:57.182 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.182 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.182 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:57.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:57.183 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:19:57.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:19:57.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:57.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:19:57.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:19:57.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:19:57.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:19:57.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:19:57.318 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:19:57.318 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:57.318 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:57.318 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:57.319 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.319 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.319 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:57.319 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:57.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:57.335 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T16:19:57.335 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:57.336 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:57.336 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:57.336 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.336 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.336 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:57.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:19:57.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:57.373 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:57.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:57.382 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:57.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:57.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-19T16:19:57.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:19:57.384 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:57.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=78026479-9667-4631-8816-7fac805b76e9 2026-02-19T16:19:57.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 78026479-9667-4631-8816-7fac805b76e9' 2026-02-19T16:19:57.384 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 78026479-9667-4631-8816-7fac805b76e9 2026-02-19T16:19:57.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:57.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAtOJdpIRMjFxAAXivzdLp8vCzIE3xnqkBOsg== 2026-02-19T16:19:57.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAtOJdpIRMjFxAAXivzdLp8vCzIE3xnqkBOsg=="}' 2026-02-19T16:19:57.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 78026479-9667-4631-8816-7fac805b76e9 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:57.493 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:57.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:57.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQAtOJdpIRMjFxAAXivzdLp8vCzIE3xnqkBOsg== --osd-uuid 78026479-9667-4631-8816-7fac805b76e9 2026-02-19T16:19:57.516 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:57.514+0000 7f8297c12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:57.518 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:57.516+0000 7f8297c12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:57.519 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:57.517+0000 7f8297c12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:57.519 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:57.517+0000 7f8297c12780 -1 bdev(0x328ac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:57.519 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:57.517+0000 7f8297c12780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:19:57.922 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:19:57.922 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:19:57.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:19:57.923 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:19:57.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:19:58.025 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:19:58.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:19:58.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-19T16:19:58.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:19:58.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:19:58.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:19:58.040 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:58.038+0000 7f1d215fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:58.041 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:58.039+0000 7f1d215fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:58.042 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:58.040+0000 7f1d215fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:58.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:19:58.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:19:58.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:19:58.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:19:58.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:19:58.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:58.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:19:58.123 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:58.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:58.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:58.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:19:58.262 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:58.261+0000 7f1d215fd780 -1 Falling back to public interface 2026-02-19T16:19:58.620 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:58.618+0000 7f1d215fd780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:19:59.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:19:59.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:19:59.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:19:59.220 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:19:59.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:19:59.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:19:59.316 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3071023411,v1:127.0.0.1:6801/3071023411] [v2:127.0.0.1:6802/3071023411,v1:127.0.0.1:6803/3071023411] exists,up 78026479-9667-4631-8816-7fac805b76e9 2026-02-19T16:19:59.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:19:59.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:19:59.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:19:59.317 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T16:19:59.317 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:19:59.317 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:19:59.318 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:59.318 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.318 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.318 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:19:59.318 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:19:59.318 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:19:59.366 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:59.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:19:59.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:19:59.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:19:59.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:59.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:59.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:59.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:59.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:59.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:59.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:59.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:59.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:59.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:59.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:59.483 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:59.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:59.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:59.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:59.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:59.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:59.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:59.486 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:59.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:59.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:59.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:59.490 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:59.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:59.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:59.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:19:59.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:19:59.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:19:59.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:19:59.494 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:19:59.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:19:59.495 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:19:59.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:19:59.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:19:59.496 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:19:59.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:59.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:19:59.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:19:59.498 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:19:59.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:19:59.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:19:59.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:19:59.500 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:19:59.500 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.500 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:19:59.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:19:59.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:19:59.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:19:59.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:19:59.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:19:59.503 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:19:59.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:19:59.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:19:59.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:19:59.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-19T16:19:59.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:59.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:19:59.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:19:59.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:19:59.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:19:59.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-19T16:19:59.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:19:59.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:59.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:59.522 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:59.522 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.522 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:59.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-19T16:19:59.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:59.544 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.545 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.545 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:59.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:59.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:59.587 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:59.588 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:59.588 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.588 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.588 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:59.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:19:59.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:19:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:19:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:19:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:19:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:19:59.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:19:59.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:19:59.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:19:59.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:59.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:59.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:59.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.727 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:59.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:59.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:19:59.743 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:19:59.780 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:19:59.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:19:59.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:19:59.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5d75b6cb-972b-450d-8ed5-a2874632e440 2026-02-19T16:19:59.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5d75b6cb-972b-450d-8ed5-a2874632e440' 2026-02-19T16:19:59.791 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 5d75b6cb-972b-450d-8ed5-a2874632e440 2026-02-19T16:19:59.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:19:59.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAvOJdp2aJwLxAA1dawaQukrIN/WeqJIcxIzQ== 2026-02-19T16:19:59.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAvOJdp2aJwLxAA1dawaQukrIN/WeqJIcxIzQ=="}' 2026-02-19T16:19:59.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5d75b6cb-972b-450d-8ed5-a2874632e440 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:59.897 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:19:59.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:19:59.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQAvOJdp2aJwLxAA1dawaQukrIN/WeqJIcxIzQ== --osd-uuid 5d75b6cb-972b-450d-8ed5-a2874632e440 2026-02-19T16:19:59.919 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:59.917+0000 7f1660635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:59.920 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:59.919+0000 7f1660635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:59.921 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:59.920+0000 7f1660635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:19:59.922 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:59.920+0000 7f1660635780 -1 bdev(0x2576c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:19:59.922 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:19:59.920+0000 7f1660635780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:00.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:00.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:00.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:00.370 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:00.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:00.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:00.470 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:00.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-19T16:20:00.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:00.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:00.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:00.485 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:00.483+0000 7fdc07426780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:00.486 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:00.485+0000 7fdc07426780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:00.487 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:00.485+0000 7fdc07426780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:00.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:00.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:00.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:00.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:00.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:00.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:00.569 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:00.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:00.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:00.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:00.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:00.700 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:00.698+0000 7fdc07426780 -1 Falling back to public interface 2026-02-19T16:20:01.015 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:01.013+0000 7fdc07426780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:01.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:01.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:01.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:01.661 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:01.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:01.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4189469689,v1:127.0.0.1:6801/4189469689] [v2:127.0.0.1:6802/4189469689,v1:127.0.0.1:6803/4189469689] exists,up 5d75b6cb-972b-450d-8ed5-a2874632e440 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:01.759 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:01.760 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:01.760 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:01.760 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:01.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T16:20:01.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:01.805 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:01.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:01.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:01.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:01.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:01.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:01.919 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:01.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:01.920 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:01.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:01.922 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:01.922 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:01.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:01.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:01.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:01.924 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:01.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:01.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:01.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:01.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:01.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:01.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:01.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:01.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:01.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:01.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:01.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:01.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:01.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:01.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:01.934 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:01.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:01.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:01.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:01.935 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:01.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:01.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:01.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:01.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:01.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:01.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:01.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:01.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:01.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:01.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:20:01.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:01.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:01.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:01.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:20:01.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:20:01.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:01.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:20:01.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:01.983 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:01.983 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:01.983 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:01.984 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:01.984 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:01.984 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:01.984 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:01.984 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:01.985 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:01.985 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:01.985 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:01.985 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:01.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:01.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:02.030 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:02.031 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:02.031 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:02.031 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:02.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-19T16:20:02.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:02.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:02.074 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:02.074 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:02.074 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:02.074 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:02.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-19T16:20:02.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:20:02.109 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:02.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:20:02.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:02.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:20:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:02.214 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:02.214 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:02.214 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:02.214 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:02.214 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:02.214 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:02.214 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:02.215 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:02.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:20:02.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:02.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:02.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:02.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:02.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:02.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:02.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:02.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:20:02.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:02.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=839482fc-48f5-41cf-9382-1350991da474 2026-02-19T16:20:02.236 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 839482fc-48f5-41cf-9382-1350991da474 2026-02-19T16:20:02.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 839482fc-48f5-41cf-9382-1350991da474' 2026-02-19T16:20:02.236 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:02.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAyOJdpVGRHDhAAqdUMszv7UsNuVBOhpBiMag== 2026-02-19T16:20:02.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAyOJdpVGRHDhAAqdUMszv7UsNuVBOhpBiMag=="}' 2026-02-19T16:20:02.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 839482fc-48f5-41cf-9382-1350991da474 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:02.342 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:02.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:02.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAyOJdpVGRHDhAAqdUMszv7UsNuVBOhpBiMag== --osd-uuid 839482fc-48f5-41cf-9382-1350991da474 2026-02-19T16:20:02.365 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.362+0000 7fb23a842780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:02.366 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.364+0000 7fb23a842780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:02.366 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.365+0000 7fb23a842780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:02.367 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.365+0000 7fb23a842780 -1 bdev(0x307cc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:02.367 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.365+0000 7fb23a842780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:02.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:02.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:02.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:02.827 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:02.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:02.929 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:02.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:02.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:02.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:02.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:02.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:02.943 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.941+0000 7fa4fc035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:02.944 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.943+0000 7fa4fc035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:02.945 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:02.944+0000 7fa4fc035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:03.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:03.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:03.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:03.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:03.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:03.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:03.026 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:03.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:03.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:03.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:03.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:03.133 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:03.132+0000 7fa4fc035780 -1 Falling back to public interface 2026-02-19T16:20:03.482 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:03.480+0000 7fa4fc035780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:04.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:04.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:04.125 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:04.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:04.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:04.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:04.221 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/245732521,v1:127.0.0.1:6801/245732521] [v2:127.0.0.1:6802/245732521,v1:127.0.0.1:6803/245732521] exists,up 839482fc-48f5-41cf-9382-1350991da474 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:04.223 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.224 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.224 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:04.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-19T16:20:04.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:04.269 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:04.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:04.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:04.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-19T16:20:04.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:04.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:04.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:04.278 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:04.278 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:04.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:04.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:04.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:04.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:04.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:04.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:04.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:04.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:04.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:04.388 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:04.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:04.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:04.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:04.389 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:04.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:04.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:04.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:04.394 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:04.394 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.394 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:04.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:04.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:04.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:04.397 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:04.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:04.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:04.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:04.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:04.399 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:04.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:04.399 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:04.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:04.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:04.400 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:04.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:04.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:04.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:04.401 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:04.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:04.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:04.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:04.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:04.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:04.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:04.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:04.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:20:04.406 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:04.406 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.406 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:20:04.407 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:04.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:20:04.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:20:04.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-19T16:20:04.426 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:04.426 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:04.426 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:04.426 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:04.426 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.426 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.426 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:04.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-19T16:20:04.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:04.451 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:04.452 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:04.452 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.452 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.452 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:04.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:04.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:20:04.494 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:04.494 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:04.495 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:04.495 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:04.495 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:04.495 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:04.495 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:04.495 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:04.496 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:04.496 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.496 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.496 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:04.496 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:04.497 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:04.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:20:04.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:04.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:04.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:04.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:04.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:20:04.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:04.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:04.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:04.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:04.640 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:04.640 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.640 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:04.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:04.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:04.656 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T16:20:04.656 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:04.656 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:04.657 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:04.657 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.657 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:04.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:04.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:20:04.695 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:04.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:04.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:04.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:04.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:04.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:04.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:04.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:20:04.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:20:04.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:04.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:04.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:04.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:04.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:04.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-19T16:20:04.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:20:04.706 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:04.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3c33f39d-caaa-4578-941d-3756f54d2e17 2026-02-19T16:20:04.707 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 3c33f39d-caaa-4578-941d-3756f54d2e17 2026-02-19T16:20:04.708 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3c33f39d-caaa-4578-941d-3756f54d2e17' 2026-02-19T16:20:04.708 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:04.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA0OJdprIJVKhAA0g1b0+n1OGFI6P8Q46MANQ== 2026-02-19T16:20:04.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA0OJdprIJVKhAA0g1b0+n1OGFI6P8Q46MANQ=="}' 2026-02-19T16:20:04.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3c33f39d-caaa-4578-941d-3756f54d2e17 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:04.810 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:04.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:04.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQA0OJdprIJVKhAA0g1b0+n1OGFI6P8Q46MANQ== --osd-uuid 3c33f39d-caaa-4578-941d-3756f54d2e17 2026-02-19T16:20:04.833 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:04.831+0000 7fcde2649780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:04.835 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:04.833+0000 7fcde2649780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:04.835 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:04.834+0000 7fcde2649780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:04.836 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:04.834+0000 7fcde2649780 -1 bdev(0x2a50c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:04.836 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:04.834+0000 7fcde2649780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:05.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:05.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:05.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:05.277 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:05.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:05.377 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:05.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:05.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-19T16:20:05.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:05.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:05.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:05.392 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:05.390+0000 7f29fce33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:05.393 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:05.391+0000 7f29fce33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:05.394 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:05.392+0000 7f29fce33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:05.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:05.604 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:05.603+0000 7f29fce33780 -1 Falling back to public interface 2026-02-19T16:20:05.938 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:05.936+0000 7f29fce33780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:06.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:06.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:06.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:06.571 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:06.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:06.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:06.667 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1817529621,v1:127.0.0.1:6801/1817529621] [v2:127.0.0.1:6802/1817529621,v1:127.0.0.1:6803/1817529621] exists,up 3c33f39d-caaa-4578-941d-3756f54d2e17 2026-02-19T16:20:06.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:06.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:06.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:06.712 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:06.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:06.718 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:04.440+0000 7f0de1c2dc00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T16:20:06.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:06.719 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:05.603+0000 7f29fce33780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T16:20:06.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:20:06.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:06.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:06.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:06.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:06.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:06.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:06.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:06.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:06.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:06.827 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:06.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:06.828 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:06.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:06.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:06.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:06.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:06.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:06.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:06.831 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:06.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:06.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:06.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:06.835 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:06.835 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:06.835 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:06.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:06.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:06.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:06.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T16:20:06.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:06.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:06.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:06.839 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:06.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:06.840 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:06.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:06.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:06.841 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:06.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:06.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:06.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:06.842 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:06.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:06.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:06.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:06.845 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:06.845 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:06.845 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:06.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:06.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:06.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:06.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:20:06.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:06.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:06.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:06.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:20:06.848 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:06.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:20:06.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:06.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:20:06.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T16:20:06.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:06.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:06.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:06.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:06.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:20:06.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T16:20:06.866 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:06.866 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:06.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:06.867 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:06.867 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:06.867 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:06.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:06.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T16:20:06.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:06.891 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:06.891 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:06.891 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:06.891 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:06.891 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:06.892 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:06.893 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:06.936 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:06.937 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:06.937 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:06.937 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:06.937 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:06.937 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:06.937 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:06.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:20:06.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:06.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:06.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:06.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:06.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:20:06.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:07.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:07.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:07.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:07.075 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:07.075 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:07.075 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:07.076 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:07.076 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:07.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:07.092 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:07.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:20:07.130 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:07.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T16:20:07.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:07.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:07.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:07.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:07.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:07.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c321d6a9-b4da-4747-b158-b3cadae3b760 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c321d6a9-b4da-4747-b158-b3cadae3b760' 2026-02-19T16:20:07.141 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 c321d6a9-b4da-4747-b158-b3cadae3b760 2026-02-19T16:20:07.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:07.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA3OJdp15apCBAA5XQ1bKbWIdsEEAgfHeOLYg== 2026-02-19T16:20:07.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA3OJdp15apCBAA5XQ1bKbWIdsEEAgfHeOLYg=="}' 2026-02-19T16:20:07.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c321d6a9-b4da-4747-b158-b3cadae3b760 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:07.249 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:07.255 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:07.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQA3OJdp15apCBAA5XQ1bKbWIdsEEAgfHeOLYg== --osd-uuid c321d6a9-b4da-4747-b158-b3cadae3b760 2026-02-19T16:20:07.271 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.269+0000 7f1c4183f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:07.272 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.271+0000 7f1c4183f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:07.273 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.271+0000 7f1c4183f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:07.274 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.272+0000 7f1c4183f780 -1 bdev(0x4136c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:07.274 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.272+0000 7f1c4183f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:07.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:07.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:07.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:07.685 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:07.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:07.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:07.787 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:07.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T16:20:07.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:07.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:07.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:07.800 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.799+0000 7f6d8c80f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:07.802 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.800+0000 7f6d8c80f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:07.803 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:07.801+0000 7f6d8c80f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:07.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:07.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:07.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:07.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:07.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:07.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:07.887 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:07.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:07.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:07.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:07.980 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:08.019 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:08.017+0000 7f6d8c80f780 -1 Falling back to public interface 2026-02-19T16:20:08.427 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:08.425+0000 7f6d8c80f780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:08.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:08.982 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:08.982 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:08.982 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:08.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:08.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:09.078 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2779869074,v1:127.0.0.1:6801/2779869074] [v2:127.0.0.1:6802/2779869074,v1:127.0.0.1:6803/2779869074] exists,up c321d6a9-b4da-4747-b158-b3cadae3b760 2026-02-19T16:20:09.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:09.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:09.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:09.123 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:09.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:09.128 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:06.880+0000 7f2160843c00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T16:20:09.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:09.130 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:08.017+0000 7f6d8c80f780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:09.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:09.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:09.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:09.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:09.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:09.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:09.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:09.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:09.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:09.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:09.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:09.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:09.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:09.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:09.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:09.247 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:09.247 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.247 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:09.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:09.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:09.250 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:09.250 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:09.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:09.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:09.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:09.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:09.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:09.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:09.252 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:09.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:09.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:09.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:09.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:09.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:09.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:09.254 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:09.255 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:09.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:09.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:09.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:09.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:09.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:09.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:09.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:20:09.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:09.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:20:09.260 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:09.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:20:09.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:20:09.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T16:20:09.278 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:09.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:09.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:09.279 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:09.279 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.279 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:09.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T16:20:09.302 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:09.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:09.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:09.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:09.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:09.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:09.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:20:09.350 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:09.350 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:09.350 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:09.350 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.351 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:09.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:09.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:09.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:20:09.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:09.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:09.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:09.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:09.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:20:09.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:09.488 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:09.488 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:09.488 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:09.488 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:09.488 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.488 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.489 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:09.489 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:09.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:09.503 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:09.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:20:09.542 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:09.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:09.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:09.551 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:09.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ef9ea491-ae2e-4739-ab5e-21b156f2120e 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ef9ea491-ae2e-4739-ab5e-21b156f2120e' 2026-02-19T16:20:09.553 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 ef9ea491-ae2e-4739-ab5e-21b156f2120e 2026-02-19T16:20:09.554 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:09.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA5OJdpomc4IRAAfBqKn1c3PKOEGiCMr7phlQ== 2026-02-19T16:20:09.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA5OJdpomc4IRAAfBqKn1c3PKOEGiCMr7phlQ=="}' 2026-02-19T16:20:09.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ef9ea491-ae2e-4739-ab5e-21b156f2120e -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:09.658 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:09.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:09.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQA5OJdpomc4IRAAfBqKn1c3PKOEGiCMr7phlQ== --osd-uuid ef9ea491-ae2e-4739-ab5e-21b156f2120e 2026-02-19T16:20:09.681 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:09.678+0000 7f39783d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:09.683 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:09.680+0000 7f39783d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:09.683 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:09.681+0000 7f39783d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:09.684 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:09.681+0000 7f39783d4780 -1 bdev(0x3488c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:09.684 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:09.681+0000 7f39783d4780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:10.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:10.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:10.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:10.062 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:10.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:10.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:10.161 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:10.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T16:20:10.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:10.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:10.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:10.175 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:10.172+0000 7faea17fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:10.176 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:10.173+0000 7faea17fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:10.177 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:10.174+0000 7faea17fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:10.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:10.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:10.388 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:10.386+0000 7faea17fd780 -1 Falling back to public interface 2026-02-19T16:20:10.736 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:10.733+0000 7faea17fd780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:11.356 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:11.356 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:11.357 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:11.357 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:11.358 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:11.358 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:11.452 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1679926869,v1:127.0.0.1:6801/1679926869] [v2:127.0.0.1:6802/1679926869,v1:127.0.0.1:6803/1679926869] exists,up ef9ea491-ae2e-4739-ab5e-21b156f2120e 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:11.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:11.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:11.498 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:11.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:11.503 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:09.291+0000 7f0d4da34c00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T16:20:11.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:11.505 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:10.386+0000 7faea17fd780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:11.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:11.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:11.612 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:11.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:11.613 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:11.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:11.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:11.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:11.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:11.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:11.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:11.617 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:11.618 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:11.618 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:11.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:11.622 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:11.622 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.622 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:11.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:11.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:11.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:11.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:11.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:11.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:11.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:11.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:11.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:11.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:11.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:11.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:11.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:11.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:11.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:11.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:11.629 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:11.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:11.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:11.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:11.631 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:11.631 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.631 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:11.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:11.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:11.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:20:11.633 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:11.634 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.634 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:20:11.635 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:11.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:20:11.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:11.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:20:11.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-19T16:20:11.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:11.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:11.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:11.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:11.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:20:11.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-19T16:20:11.653 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:11.653 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:11.653 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:11.653 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:11.654 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.654 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.654 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:11.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-19T16:20:11.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:11.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:11.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:11.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:11.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:11.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:11.679 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:11.679 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:11.679 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:11.679 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:11.679 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.680 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.680 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:11.680 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:11.680 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:11.725 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:11.726 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:11.726 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.726 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:11.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:11.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:20:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:20:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:11.866 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:11.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:11.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:11.867 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:11.867 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.867 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:11.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:11.868 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:11.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T16:20:11.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:11.883 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:11.883 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:11.883 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.883 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.883 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:11.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:11.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:20:11.921 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:11.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:20:11.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:11.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4fd851af-8111-47ba-a3dd-382b6f7f7614 2026-02-19T16:20:11.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4fd851af-8111-47ba-a3dd-382b6f7f7614' 2026-02-19T16:20:11.928 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 4fd851af-8111-47ba-a3dd-382b6f7f7614 2026-02-19T16:20:11.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:11.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA7OJdpT27JNxAAQ9AublCknbDIrhzFZ6P5Lg== 2026-02-19T16:20:11.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA7OJdpT27JNxAAQ9AublCknbDIrhzFZ6P5Lg=="}' 2026-02-19T16:20:11.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4fd851af-8111-47ba-a3dd-382b6f7f7614 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:12.040 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:12.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:12.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQA7OJdpT27JNxAAQ9AublCknbDIrhzFZ6P5Lg== --osd-uuid 4fd851af-8111-47ba-a3dd-382b6f7f7614 2026-02-19T16:20:12.064 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.061+0000 7f441a25d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:12.065 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.062+0000 7f441a25d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:12.066 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.063+0000 7f441a25d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:12.067 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.063+0000 7f441a25d780 -1 bdev(0x2a26c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:12.067 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.064+0000 7f441a25d780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:12.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:12.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:12.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:12.482 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:12.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:12.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:12.584 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:12.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-19T16:20:12.585 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:12.585 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:12.585 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:12.598 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.596+0000 7f84985d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:12.600 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.597+0000 7f84985d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:12.601 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.598+0000 7f84985d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:12.680 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:12.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:12.817 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:12.814+0000 7f84985d8780 -1 Falling back to public interface 2026-02-19T16:20:13.186 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:13.183+0000 7f84985d8780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:13.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:13.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:13.778 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:13.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:13.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:13.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:13.872 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/918938436,v1:127.0.0.1:6801/918938436] [v2:127.0.0.1:6802/918938436,v1:127.0.0.1:6803/918938436] exists,up 4fd851af-8111-47ba-a3dd-382b6f7f7614 2026-02-19T16:20:13.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:13.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:13.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:13.920 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:13.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:13.927 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:11.666+0000 7fe40447dc00 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T16:20:13.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:13.928 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:12.815+0000 7f84985d8780 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T16:20:13.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:20:13.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:13.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:13.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:13.929 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:13.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:13.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:13.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:13.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:14.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:14.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:14.037 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:14.037 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:14.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:14.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:14.039 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:14.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:14.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:14.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:14.041 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:14.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:14.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:14.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:14.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:14.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:14.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:14.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:14.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T16:20:14.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T16:20:14.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:14.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:20:14.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:14.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:14.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:14.048 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:14.048 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:14.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:14.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:14.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:14.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:14.049 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:14.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:14.049 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:14.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:14.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:14.050 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:14.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:14.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:14.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:14.052 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:14.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:14.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:14.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:14.054 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:14.054 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.055 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:14.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:14.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:14.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:20:14.057 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:14.057 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.057 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:20:14.059 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:14.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:20:14.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:14.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:20:14.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-19T16:20:14.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:14.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:14.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:14.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:14.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:20:14.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-19T16:20:14.077 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:14.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:14.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:14.078 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:14.078 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.078 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:14.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-19T16:20:14.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:14.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:14.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:14.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:14.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:14.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:14.105 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:14.106 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:14.149 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:14.150 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:14.150 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.150 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.150 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:14.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:14.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:14.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:20:14.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:14.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:14.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:14.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:14.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:20:14.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:14.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:14.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:14.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:14.291 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:14.291 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.291 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.292 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:14.292 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:14.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:14.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T16:20:14.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:14.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:14.308 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:14.308 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.308 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:14.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:14.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:20:14.345 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-19T16:20:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:14.353 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:14.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:14.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:14.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:14.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-19T16:20:14.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:20:14.355 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:14.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6c896554-4eff-4d77-a2f8-acd3d7d98de5 2026-02-19T16:20:14.355 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 6c896554-4eff-4d77-a2f8-acd3d7d98de5 2026-02-19T16:20:14.356 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6c896554-4eff-4d77-a2f8-acd3d7d98de5' 2026-02-19T16:20:14.356 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:14.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+OJdpdGN0FRAAMApKsHkDSVqHGIS5ifj4pA== 2026-02-19T16:20:14.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+OJdpdGN0FRAAMApKsHkDSVqHGIS5ifj4pA=="}' 2026-02-19T16:20:14.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6c896554-4eff-4d77-a2f8-acd3d7d98de5 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:14.464 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:14.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:14.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQA+OJdpdGN0FRAAMApKsHkDSVqHGIS5ifj4pA== --osd-uuid 6c896554-4eff-4d77-a2f8-acd3d7d98de5 2026-02-19T16:20:14.487 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:14.484+0000 7f9902bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:14.488 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:14.485+0000 7f9902bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:14.489 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:14.486+0000 7f9902bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:14.490 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:14.486+0000 7f9902bfd780 -1 bdev(0x2d4ac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:14.490 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:14.487+0000 7f9902bfd780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:14.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:14.943 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:14.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:15.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:15.044 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:15.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-19T16:20:15.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:15.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:15.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:15.058 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:15.055+0000 7f1ee4448780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:15.059 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:15.056+0000 7f1ee4448780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:15.060 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:15.057+0000 7f1ee4448780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:15.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:15.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:15.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:15.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:15.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:15.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:15.144 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:15.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:15.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:15.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:15.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:15.265 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:15.262+0000 7f1ee4448780 -1 Falling back to public interface 2026-02-19T16:20:15.626 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:15.623+0000 7f1ee4448780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:16.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:16.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:16.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:16.241 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:16.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:16.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:16.339 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2008400454,v1:127.0.0.1:6801/2008400454] [v2:127.0.0.1:6802/2008400454,v1:127.0.0.1:6803/2008400454] exists,up 6c896554-4eff-4d77-a2f8-acd3d7d98de5 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:16.341 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.342 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.342 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:16.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:16.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:16.385 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:16.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:16.390 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:14.091+0000 7fcacae32c00 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T16:20:16.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:16.392 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:15.262+0000 7f1ee4448780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:16.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:16.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:16.500 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:16.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:16.500 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:16.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:16.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:16.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:16.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:16.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:16.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:16.504 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:16.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:16.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:16.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:16.508 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:16.508 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.508 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.508 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:16.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:16.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:16.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T16:20:16.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T16:20:16.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:16.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:16.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:16.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:16.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:16.513 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:16.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:16.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:16.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:16.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:16.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:16.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:16.515 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:16.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:16.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:16.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:16.518 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:16.518 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.518 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:16.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:16.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:16.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T16:20:16.520 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:16.520 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.520 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37597 2026-02-19T16:20:16.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:16.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T16:20:16.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-19T16:20:16.541 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:16.541 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:16.541 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:16.541 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:16.541 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.542 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.542 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:16.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-19T16:20:16.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:16.566 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:16.566 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.567 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.568 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:16.568 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:16.568 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get fsid 2026-02-19T16:20:16.610 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:16.610 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:16.610 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:16.610 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:16.611 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:16.611 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:16.611 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:16.611 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:16.612 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:16.612 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.612 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.612 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:16.612 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:16.612 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37597/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:16.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T16:20:16.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:16.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:16.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:16.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:16.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T16:20:16.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:16.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:16.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:16.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:16.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:16.753 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.753 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.753 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:16.753 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:16.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-mon.a.asok 2026-02-19T16:20:16.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:16.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-mon.a.asok log flush 2026-02-19T16:20:16.809 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:16.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-19T16:20:16.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:16.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:16.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:16.817 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:16.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:16.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=43a7204c-4252-4cb7-ac1b-7432ae0ad961 2026-02-19T16:20:16.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 43a7204c-4252-4cb7-ac1b-7432ae0ad961' 2026-02-19T16:20:16.820 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 43a7204c-4252-4cb7-ac1b-7432ae0ad961 2026-02-19T16:20:16.820 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:16.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBAOJdpEYQVMRAA2u2lcq/E7Iv4X/oinnBZHg== 2026-02-19T16:20:16.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBAOJdpEYQVMRAA2u2lcq/E7Iv4X/oinnBZHg=="}' 2026-02-19T16:20:16.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 43a7204c-4252-4cb7-ac1b-7432ae0ad961 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:16.926 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:16.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T16:20:16.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQBAOJdpEYQVMRAA2u2lcq/E7Iv4X/oinnBZHg== --osd-uuid 43a7204c-4252-4cb7-ac1b-7432ae0ad961 2026-02-19T16:20:16.949 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:16.946+0000 7f451b03c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:16.950 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:16.948+0000 7f451b03c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:16.951 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:16.948+0000 7f451b03c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:16.952 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:16.949+0000 7f451b03c780 -1 bdev(0x3494c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:16.952 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:16.949+0000 7f451b03c780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T16:20:17.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T16:20:17.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:17.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:17.377 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:17.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:17.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:17.477 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:17.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9eb29485-a905-446a-a269-cd3898951029 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.37597/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-19T16:20:17.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:17.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:17.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:17.491 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:17.488+0000 7fb88d035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:17.492 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:17.490+0000 7fb88d035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:17.493 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:17.490+0000 7fb88d035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:17.573 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:17.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:17.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:17.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:17.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:17.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:17.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:17.575 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:17.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:17.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:17.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:17.692 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:17.689+0000 7fb88d035780 -1 Falling back to public interface 2026-02-19T16:20:18.046 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:18.043+0000 7fb88d035780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:18.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:18.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:18.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:18.672 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:18.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:18.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:18.767 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2967306845,v1:127.0.0.1:6801/2967306845] [v2:127.0.0.1:6802/2967306845,v1:127.0.0.1:6803/2967306845] exists,up 43a7204c-4252-4cb7-ac1b-7432ae0ad961 2026-02-19T16:20:18.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:18.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:18.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:18.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T16:20:18.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:18.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:18.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:18.770 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:18.770 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:18.770 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37597/ceph-osd.0.asok 2026-02-19T16:20:18.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T16:20:18.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.37597/ceph-osd.0.asok log flush 2026-02-19T16:20:18.816 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:18.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T16:20:18.821 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:16.554+0000 7fe67c28ec00 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T16:20:18.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-19T16:20:18.823 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:17.690+0000 7fb88d035780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:18.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:18.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:18.931 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:18.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:18.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:18.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:18.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:18.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:18.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:18.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:18.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:18.935 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:18.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:18.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:18.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:18.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:18.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:18.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:18.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code-plugins 0 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:18.942 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:18.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:18.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:18.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:18.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:18.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:18.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:18.944 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:18.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:18.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:18.946 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:18.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:18.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:18.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:18.947 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:18.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:18.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T16:20:18.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T16:20:18.950 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:18.950 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:18.950 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37597 2026-02-19T16:20:18.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37597 2026-02-19T16:20:18.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:18.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:18.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-19T16:20:18.952 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T16:20:18.952 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T16:20:19.018 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-19T16:20:19.018 DEBUG:teuthology.orchestra.run.trial127:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=802bf111cd55e57f5842153d7ba2b8b964aa818a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-19T16:20:19.081 INFO:tasks.workunit.client.0.trial127.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T16:20:19.086 INFO:tasks.workunit.client.0.trial127.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T16:20:19.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-19T16:20:19.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none ' 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-19T16:20:19.088 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:20:19.089 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:20:19.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:20:19.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:20:19.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:20:19.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:20:19.090 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:20:19.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:20:19.090 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:20:19.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:20:19.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:20:19.091 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:20:19.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:19.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:20:19.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:20:19.092 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:20:19.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:20:19.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:20:19.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-19T16:20:19.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:20:19.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.094 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.094 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.67892 2026-02-19T16:20:19.094 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:20:19.094 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:20:19.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-19T16:20:19.095 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:20:19.095 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.096 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.67892 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:20:19.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-19T16:20:19.099 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:19.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:20:19.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:20:19.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:20:19.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:19.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:19.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:20:19.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:20:19.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:19.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:19.142 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:19.142 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:19.142 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.142 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.142 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.67892/ceph-mon.a.asok 2026-02-19T16:20:19.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:19.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.67892/ceph-mon.a.asok config get fsid 2026-02-19T16:20:19.188 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:20:19.188 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:20:19.188 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:20:19.188 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:20:19.188 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:20:19.189 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:20:19.189 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:19.189 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:19.189 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:19.189 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.189 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.189 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.67892/ceph-mon.a.asok 2026-02-19T16:20:19.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:20:19.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.67892/ceph-mon.a.asok config get mon_host 2026-02-19T16:20:19.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-19T16:20:19.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-19T16:20:19.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:20:19.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:20:19.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:20:19.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-19T16:20:19.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:20:19.328 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:20:19.328 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:19.329 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:19.329 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:19.329 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.329 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.329 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:19.329 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:19.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:20:19.345 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-19T16:20:19.345 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:20:19.345 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:20:19.345 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:19.346 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.346 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.346 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.67892/ceph-mon.a.asok 2026-02-19T16:20:19.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-19T16:20:19.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.67892/ceph-mon.a.asok log flush 2026-02-19T16:20:19.385 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:19.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-19T16:20:19.390 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:19.129+0000 7f559b02dc00 0 load: jerasure load: lrc 2026-02-19T16:20:19.390 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-19T16:20:19.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:19.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-19T16:20:19.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:19.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:19.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:20:19.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:19.395 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:19.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=00fa9a63-a942-4122-8826-f23b089bc36b 2026-02-19T16:20:19.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 00fa9a63-a942-4122-8826-f23b089bc36b' 2026-02-19T16:20:19.398 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 00fa9a63-a942-4122-8826-f23b089bc36b 2026-02-19T16:20:19.398 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:19.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDOJdpMMn6FxAAciPbFrsuxVownSKRreSnPw== 2026-02-19T16:20:19.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDOJdpMMn6FxAAciPbFrsuxVownSKRreSnPw=="}' 2026-02-19T16:20:19.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 00fa9a63-a942-4122-8826-f23b089bc36b -i td/test-erasure-code/0/new.json 2026-02-19T16:20:19.505 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:19.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-19T16:20:19.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDOJdpMMn6FxAAciPbFrsuxVownSKRreSnPw== --osd-uuid 00fa9a63-a942-4122-8826-f23b089bc36b 2026-02-19T16:20:19.528 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:19.527+0000 7fa30013d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:19.530 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:19.528+0000 7fa30013d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:19.531 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:19.529+0000 7fa30013d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:19.531 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:19.529+0000 7fa30013d780 -1 bdev(0x372ec00 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:19.531 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:19.529+0000 7fa30013d780 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-19T16:20:19.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-19T16:20:19.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:19.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:20:19.931 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:20:19.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:20.032 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:20:20.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:20:20.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:20.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:20.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:20.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:20.045 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:20.044+0000 7f1d4b244780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:20.047 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:20.045+0000 7f1d4b244780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:20.047 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:20.046+0000 7f1d4b244780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:20.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:20:20.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:20.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:20:20.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:20.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:20.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:20.130 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:20.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:20.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:20.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:20.226 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:20.223+0000 7f1d4b244780 -1 Falling back to public interface 2026-02-19T16:20:20.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:20.563 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:20.561+0000 7f1d4b244780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:20:21.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:21.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:21.228 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:21.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:21.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:21.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:20:21.325 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1676673719,v1:127.0.0.1:6801/1676673719] [v2:127.0.0.1:6802/1676673719,v1:127.0.0.1:6803/1676673719] exists,up 00fa9a63-a942-4122-8826-f23b089bc36b 2026-02-19T16:20:21.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:21.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:21.330 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:21.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:21.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-19T16:20:21.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:21.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e3fbdca-f78b-4555-90d8-e1d66645a61c 2026-02-19T16:20:21.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4e3fbdca-f78b-4555-90d8-e1d66645a61c' 2026-02-19T16:20:21.332 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 4e3fbdca-f78b-4555-90d8-e1d66645a61c 2026-02-19T16:20:21.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:21.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBFOJdpymgOFBAAVmAulmy4jOq/qT284WSOJw== 2026-02-19T16:20:21.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBFOJdpymgOFBAAVmAulmy4jOq/qT284WSOJw=="}' 2026-02-19T16:20:21.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e3fbdca-f78b-4555-90d8-e1d66645a61c -i td/test-erasure-code/1/new.json 2026-02-19T16:20:21.440 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:21.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-19T16:20:21.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBFOJdpymgOFBAAVmAulmy4jOq/qT284WSOJw== --osd-uuid 4e3fbdca-f78b-4555-90d8-e1d66645a61c 2026-02-19T16:20:21.461 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:21.460+0000 7f61f8c64780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:21.463 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:21.461+0000 7f61f8c64780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:21.464 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:21.462+0000 7f61f8c64780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:21.464 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:21.462+0000 7f61f8c64780 -1 bdev(0x4410c00 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:21.464 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:21.462+0000 7f61f8c64780 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-19T16:20:21.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-19T16:20:21.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:21.780 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:20:21.780 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:20:21.780 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:22.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:20:22.097 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:20:22.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:22.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:22.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:22.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:22.110 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:22.108+0000 7f535e62c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:22.112 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:22.110+0000 7f535e62c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:22.112 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:22.111+0000 7f535e62c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:22.241 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:22.239+0000 7f535e62c780 -1 Falling back to public interface 2026-02-19T16:20:22.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:20:22.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:22.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:20:22.544 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:22.542+0000 7f535e62c780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:20:22.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:23.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:23.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:23.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:23.550 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:23.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:23.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:20:23.777 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/542609970,v1:127.0.0.1:6811/542609970] [v2:127.0.0.1:6812/542609970,v1:127.0.0.1:6813/542609970] exists,up 4e3fbdca-f78b-4555-90d8-e1d66645a61c 2026-02-19T16:20:23.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:23.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:23.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:23.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:23.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:23.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-19T16:20:23.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:23.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7449dcf1-7892-4daf-aa48-dcc46491f8b1 2026-02-19T16:20:23.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 7449dcf1-7892-4daf-aa48-dcc46491f8b1' 2026-02-19T16:20:23.785 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 7449dcf1-7892-4daf-aa48-dcc46491f8b1 2026-02-19T16:20:23.785 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:23.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBHOJdpoiUMLxAAXuBBSW2xiqRoyVIOS2yTDA== 2026-02-19T16:20:23.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBHOJdpoiUMLxAAXuBBSW2xiqRoyVIOS2yTDA=="}' 2026-02-19T16:20:23.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7449dcf1-7892-4daf-aa48-dcc46491f8b1 -i td/test-erasure-code/2/new.json 2026-02-19T16:20:24.020 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:20:24.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-19T16:20:24.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBHOJdpoiUMLxAAXuBBSW2xiqRoyVIOS2yTDA== --osd-uuid 7449dcf1-7892-4daf-aa48-dcc46491f8b1 2026-02-19T16:20:24.044 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.042+0000 7f3380e44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:24.045 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.043+0000 7f3380e44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:24.046 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.044+0000 7f3380e44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:24.046 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.044+0000 7f3380e44780 -1 bdev(0x2d44c00 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:24.046 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.044+0000 7f3380e44780 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-19T16:20:24.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-19T16:20:24.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:24.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:20:24.370 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:20:24.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:24.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:20:24.685 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:20:24.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:24.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:24.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:24.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:24.700 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.698+0000 7f3d91033780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:24.701 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.700+0000 7f3d91033780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:24.702 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.700+0000 7f3d91033780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:24.889 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:24.886+0000 7f3d91033780 -1 Falling back to public interface 2026-02-19T16:20:24.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:20:24.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:24.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:20:24.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:24.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:24.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:24.914 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:24.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:24.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:24.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:20:25.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:25.258 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:25.256+0000 7f3d91033780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:20:26.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:26.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:26.137 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:26.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:26.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:26.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:20:26.363 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1441892163,v1:127.0.0.1:6819/1441892163] [v2:127.0.0.1:6820/1441892163,v1:127.0.0.1:6821/1441892163] exists,up 7449dcf1-7892-4daf-aa48-dcc46491f8b1 2026-02-19T16:20:26.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:26.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-19T16:20:26.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:26.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-19T16:20:26.368 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:26.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e5db492a-74d3-4383-8045-6600e28aab52 2026-02-19T16:20:26.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e5db492a-74d3-4383-8045-6600e28aab52' 2026-02-19T16:20:26.369 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 e5db492a-74d3-4383-8045-6600e28aab52 2026-02-19T16:20:26.369 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:26.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBKOJdprzlTFhAAAQub92FesJDqcA2xLiu/Jw== 2026-02-19T16:20:26.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBKOJdprzlTFhAAAQub92FesJDqcA2xLiu/Jw=="}' 2026-02-19T16:20:26.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e5db492a-74d3-4383-8045-6600e28aab52 -i td/test-erasure-code/3/new.json 2026-02-19T16:20:26.615 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:20:26.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-19T16:20:26.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBKOJdprzlTFhAAAQub92FesJDqcA2xLiu/Jw== --osd-uuid e5db492a-74d3-4383-8045-6600e28aab52 2026-02-19T16:20:26.638 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:26.637+0000 7f4e5dc47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:26.640 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:26.638+0000 7f4e5dc47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:26.641 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:26.639+0000 7f4e5dc47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:26.641 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:26.639+0000 7f4e5dc47780 -1 bdev(0x33d2c00 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:26.641 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:26.639+0000 7f4e5dc47780 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-19T16:20:26.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-19T16:20:26.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:26.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:20:26.963 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:20:26.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:27.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:20:27.281 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:20:27.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:27.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:27.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:27.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:27.295 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:27.293+0000 7f991962f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:27.296 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:27.295+0000 7f991962f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:27.297 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:27.296+0000 7f991962f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:27.437 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:27.435+0000 7f991962f780 -1 Falling back to public interface 2026-02-19T16:20:27.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:20:27.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:27.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:20:27.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:27.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:27.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:27.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:27.504 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:27.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:27.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:20:27.727 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:27.725+0000 7f991962f780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:20:27.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:28.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:28.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:28.731 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:28.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:28.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:28.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:20:28.960 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/359254437,v1:127.0.0.1:6827/359254437] [v2:127.0.0.1:6828/359254437,v1:127.0.0.1:6829/359254437] exists,up e5db492a-74d3-4383-8045-6600e28aab52 2026-02-19T16:20:28.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:28.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:28.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:28.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-19T16:20:28.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:28.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-19T16:20:28.967 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:28.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e015f96b-a6f8-4296-b262-4380b70b695c 2026-02-19T16:20:28.968 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 e015f96b-a6f8-4296-b262-4380b70b695c 2026-02-19T16:20:28.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 e015f96b-a6f8-4296-b262-4380b70b695c' 2026-02-19T16:20:28.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:28.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBMOJdp+6r8ORAA2kSbRQdDIacN7uhqv04CMQ== 2026-02-19T16:20:28.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBMOJdp+6r8ORAA2kSbRQdDIacN7uhqv04CMQ=="}' 2026-02-19T16:20:28.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e015f96b-a6f8-4296-b262-4380b70b695c -i td/test-erasure-code/4/new.json 2026-02-19T16:20:29.206 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:20:29.213 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-19T16:20:29.214 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBMOJdp+6r8ORAA2kSbRQdDIacN7uhqv04CMQ== --osd-uuid e015f96b-a6f8-4296-b262-4380b70b695c 2026-02-19T16:20:29.229 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.227+0000 7f2a7e40e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:29.230 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.229+0000 7f2a7e40e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:29.231 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.229+0000 7f2a7e40e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:29.232 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.230+0000 7f2a7e40e780 -1 bdev(0x32dcc00 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:29.232 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.230+0000 7f2a7e40e780 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-19T16:20:29.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-19T16:20:29.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:29.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:20:29.572 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:20:29.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:29.891 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:20:29.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:20:29.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:29.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:29.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:29.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:29.906 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.904+0000 7fd7bbc41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:29.907 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.906+0000 7fd7bbc41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:29.908 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:29.907+0000 7fd7bbc41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:30.067 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:30.065+0000 7fd7bbc41780 -1 Falling back to public interface 2026-02-19T16:20:30.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:20:30.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:30.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:20:30.124 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:30.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:30.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:30.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:30.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:30.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:30.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:20:30.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:30.382 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:30.380+0000 7fd7bbc41780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:20:31.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:31.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:31.348 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:31.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:31.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:31.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:20:31.573 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/4066463894,v1:127.0.0.1:6835/4066463894] [v2:127.0.0.1:6836/4066463894,v1:127.0.0.1:6837/4066463894] exists,up e015f96b-a6f8-4296-b262-4380b70b695c 2026-02-19T16:20:31.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:31.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:31.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:31.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:31.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-19T16:20:31.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:31.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d4538fb7-dda6-45d2-9e55-e3cb1da6a649 2026-02-19T16:20:31.581 INFO:tasks.workunit.client.0.trial127.stdout:add osd5 d4538fb7-dda6-45d2-9e55-e3cb1da6a649 2026-02-19T16:20:31.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 d4538fb7-dda6-45d2-9e55-e3cb1da6a649' 2026-02-19T16:20:31.581 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:31.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBPOJdp2ujYIhAAB5I5HO50suAe7VxcLfMV4Q== 2026-02-19T16:20:31.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBPOJdp2ujYIhAAB5I5HO50suAe7VxcLfMV4Q=="}' 2026-02-19T16:20:31.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d4538fb7-dda6-45d2-9e55-e3cb1da6a649 -i td/test-erasure-code/5/new.json 2026-02-19T16:20:31.815 INFO:tasks.workunit.client.0.trial127.stdout:5 2026-02-19T16:20:31.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-19T16:20:31.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBPOJdp2ujYIhAAB5I5HO50suAe7VxcLfMV4Q== --osd-uuid d4538fb7-dda6-45d2-9e55-e3cb1da6a649 2026-02-19T16:20:31.839 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:31.837+0000 7f0c23027780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:31.840 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:31.838+0000 7f0c23027780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:31.841 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:31.839+0000 7f0c23027780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:31.841 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:31.839+0000 7f0c23027780 -1 bdev(0x272ec00 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:31.841 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:31.839+0000 7f0c23027780 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-19T16:20:32.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-19T16:20:32.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:32.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T16:20:32.169 INFO:tasks.workunit.client.0.trial127.stdout:adding osd5 key to auth repository 2026-02-19T16:20:32.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:32.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T16:20:32.486 INFO:tasks.workunit.client.0.trial127.stdout:start osd.5 2026-02-19T16:20:32.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:32.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:32.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:32.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:32.500 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:32.498+0000 7fe9f8883780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:32.501 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:32.499+0000 7fe9f8883780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:32.502 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:32.500+0000 7fe9f8883780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:32.677 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:32.675+0000 7fe9f8883780 -1 Falling back to public interface 2026-02-19T16:20:32.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T16:20:32.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:32.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T16:20:32.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:32.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:32.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:32.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:32.711 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:32.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:32.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:20:32.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:32.990 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:32.988+0000 7fe9f8883780 -1 osd.5 0 log_to_monitors true 2026-02-19T16:20:33.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:33.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:33.935 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:33.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:33.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:33.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:20:34.161 INFO:tasks.workunit.client.0.trial127.stdout:osd.5 up in weight 1 up_from 28 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3357604113,v1:127.0.0.1:6843/3357604113] [v2:127.0.0.1:6844/3357604113,v1:127.0.0.1:6845/3357604113] exists,up d4538fb7-dda6-45d2-9e55-e3cb1da6a649 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:34.162 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:34.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-19T16:20:34.164 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:34.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=373975fc-f0a9-4f3b-a655-bd5efae5a0c3 2026-02-19T16:20:34.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 373975fc-f0a9-4f3b-a655-bd5efae5a0c3' 2026-02-19T16:20:34.165 INFO:tasks.workunit.client.0.trial127.stdout:add osd6 373975fc-f0a9-4f3b-a655-bd5efae5a0c3 2026-02-19T16:20:34.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:34.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBSOJdpIGJVChAAAlZEOSLoQCk6pgFFpVsztQ== 2026-02-19T16:20:34.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBSOJdpIGJVChAAAlZEOSLoQCk6pgFFpVsztQ=="}' 2026-02-19T16:20:34.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 373975fc-f0a9-4f3b-a655-bd5efae5a0c3 -i td/test-erasure-code/6/new.json 2026-02-19T16:20:34.404 INFO:tasks.workunit.client.0.trial127.stdout:6 2026-02-19T16:20:34.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-19T16:20:34.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBSOJdpIGJVChAAAlZEOSLoQCk6pgFFpVsztQ== --osd-uuid 373975fc-f0a9-4f3b-a655-bd5efae5a0c3 2026-02-19T16:20:34.428 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:34.426+0000 7f1fa9231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:34.429 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:34.428+0000 7f1fa9231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:34.430 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:34.428+0000 7f1fa9231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:34.431 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:34.429+0000 7f1fa9231780 -1 bdev(0x349ac00 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:34.431 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:34.429+0000 7f1fa9231780 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-19T16:20:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-19T16:20:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T16:20:34.758 INFO:tasks.workunit.client.0.trial127.stdout:adding osd6 key to auth repository 2026-02-19T16:20:34.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:35.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T16:20:35.079 INFO:tasks.workunit.client.0.trial127.stdout:start osd.6 2026-02-19T16:20:35.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:35.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:35.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:35.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:35.094 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:35.093+0000 7f8d46c0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:35.096 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:35.094+0000 7f8d46c0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:35.096 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:35.095+0000 7f8d46c0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:35.230 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:35.229+0000 7f8d46c0d780 -1 Falling back to public interface 2026-02-19T16:20:35.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T16:20:35.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:35.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T16:20:35.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:35.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:35.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:35.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:35.310 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:35.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:35.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:20:35.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:35.569 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:35.567+0000 7f8d46c0d780 -1 osd.6 0 log_to_monitors true 2026-02-19T16:20:36.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:36.538 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:36.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:36.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:36.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:36.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:20:36.763 INFO:tasks.workunit.client.0.trial127.stdout:osd.6 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1646756041,v1:127.0.0.1:6851/1646756041] [v2:127.0.0.1:6852/1646756041,v1:127.0.0.1:6853/1646756041] exists,up 373975fc-f0a9-4f3b-a655-bd5efae5a0c3 2026-02-19T16:20:36.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:36.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:36.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:36.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:36.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-19T16:20:36.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:36.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:36.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:36.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:36.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c801565d-2362-4787-be66-969a05db0a5d 2026-02-19T16:20:36.771 INFO:tasks.workunit.client.0.trial127.stdout:add osd7 c801565d-2362-4787-be66-969a05db0a5d 2026-02-19T16:20:36.771 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 c801565d-2362-4787-be66-969a05db0a5d' 2026-02-19T16:20:36.771 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:36.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBUOJdp/hM7LhAAsmKyWKNnMOJ9Ackg5ky3nQ== 2026-02-19T16:20:36.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBUOJdp/hM7LhAAsmKyWKNnMOJ9Ackg5ky3nQ=="}' 2026-02-19T16:20:36.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c801565d-2362-4787-be66-969a05db0a5d -i td/test-erasure-code/7/new.json 2026-02-19T16:20:37.007 INFO:tasks.workunit.client.0.trial127.stdout:7 2026-02-19T16:20:37.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-19T16:20:37.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBUOJdp/hM7LhAAsmKyWKNnMOJ9Ackg5ky3nQ== --osd-uuid c801565d-2362-4787-be66-969a05db0a5d 2026-02-19T16:20:37.032 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.030+0000 7f1ce985a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:37.033 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.031+0000 7f1ce985a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:37.034 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.032+0000 7f1ce985a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:37.035 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.033+0000 7f1ce985a780 -1 bdev(0x40b0c00 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:37.035 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.033+0000 7f1ce985a780 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-19T16:20:37.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-19T16:20:37.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:37.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-19T16:20:37.385 INFO:tasks.workunit.client.0.trial127.stdout:adding osd7 key to auth repository 2026-02-19T16:20:37.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:37.707 INFO:tasks.workunit.client.0.trial127.stdout:start osd.7 2026-02-19T16:20:37.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-19T16:20:37.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:37.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:37.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:37.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:37.722 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.720+0000 7efd96039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:37.723 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.721+0000 7efd96039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:37.724 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.722+0000 7efd96039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:37.933 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:37.931+0000 7efd96039780 -1 Falling back to public interface 2026-02-19T16:20:37.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-19T16:20:37.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:37.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-19T16:20:37.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:37.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:37.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:37.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:37.937 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:37.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:37.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T16:20:38.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:38.213 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:38.211+0000 7efd96039780 -1 osd.7 0 log_to_monitors true 2026-02-19T16:20:39.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:39.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:39.171 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:39.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:39.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:39.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T16:20:39.398 INFO:tasks.workunit.client.0.trial127.stdout:osd.7 up in weight 1 up_from 38 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/2995581330,v1:127.0.0.1:6859/2995581330] [v2:127.0.0.1:6860/2995581330,v1:127.0.0.1:6861/2995581330] exists,up c801565d-2362-4787-be66-969a05db0a5d 2026-02-19T16:20:39.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:39.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:39.404 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:39.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:39.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:39.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-19T16:20:39.406 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:39.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dd6a2208-b467-460d-9450-f535f8e57e2b 2026-02-19T16:20:39.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 dd6a2208-b467-460d-9450-f535f8e57e2b' 2026-02-19T16:20:39.406 INFO:tasks.workunit.client.0.trial127.stdout:add osd8 dd6a2208-b467-460d-9450-f535f8e57e2b 2026-02-19T16:20:39.407 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:39.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBXOJdpvCp7GBAA9CCHdiugSN2YMQ4YbMAiPQ== 2026-02-19T16:20:39.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBXOJdpvCp7GBAA9CCHdiugSN2YMQ4YbMAiPQ=="}' 2026-02-19T16:20:39.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dd6a2208-b467-460d-9450-f535f8e57e2b -i td/test-erasure-code/8/new.json 2026-02-19T16:20:39.642 INFO:tasks.workunit.client.0.trial127.stdout:8 2026-02-19T16:20:39.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-19T16:20:39.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBXOJdpvCp7GBAA9CCHdiugSN2YMQ4YbMAiPQ== --osd-uuid dd6a2208-b467-460d-9450-f535f8e57e2b 2026-02-19T16:20:39.665 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:39.663+0000 7fbcf7451780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:39.666 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:39.664+0000 7fbcf7451780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:39.667 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:39.665+0000 7fbcf7451780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:39.667 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:39.665+0000 7fbcf7451780 -1 bdev(0x3278c00 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:39.667 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:39.665+0000 7fbcf7451780 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-19T16:20:39.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-19T16:20:39.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:39.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-19T16:20:39.992 INFO:tasks.workunit.client.0.trial127.stdout:adding osd8 key to auth repository 2026-02-19T16:20:39.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:40.305 INFO:tasks.workunit.client.0.trial127.stdout:start osd.8 2026-02-19T16:20:40.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-19T16:20:40.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:40.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:40.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:40.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:40.319 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:40.317+0000 7ff8f3c72780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:40.320 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:40.318+0000 7ff8f3c72780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:40.321 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:40.319+0000 7ff8f3c72780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:40.461 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:40.459+0000 7ff8f3c72780 -1 Falling back to public interface 2026-02-19T16:20:40.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-19T16:20:40.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:40.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-19T16:20:40.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:40.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:40.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:40.533 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:40.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:40.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:40.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T16:20:40.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:40.782 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:40.780+0000 7ff8f3c72780 -1 osd.8 0 log_to_monitors true 2026-02-19T16:20:41.763 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:41.763 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:41.763 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:41.763 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:41.763 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:41.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T16:20:41.995 INFO:tasks.workunit.client.0.trial127.stdout:osd.8 up in weight 1 up_from 43 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/4283039800,v1:127.0.0.1:6867/4283039800] [v2:127.0.0.1:6868/4283039800,v1:127.0.0.1:6869/4283039800] exists,up dd6a2208-b467-460d-9450-f535f8e57e2b 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:41.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:41.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:41.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:41.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-19T16:20:41.998 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:41.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=90109f52-d38c-40fb-b7d1-8891503f9349 2026-02-19T16:20:41.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 90109f52-d38c-40fb-b7d1-8891503f9349' 2026-02-19T16:20:41.999 INFO:tasks.workunit.client.0.trial127.stdout:add osd9 90109f52-d38c-40fb-b7d1-8891503f9349 2026-02-19T16:20:42.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:42.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBaOJdpao9+ABAAx391DZQOwR4ViaqlQbI7rg== 2026-02-19T16:20:42.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBaOJdpao9+ABAAx391DZQOwR4ViaqlQbI7rg=="}' 2026-02-19T16:20:42.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 90109f52-d38c-40fb-b7d1-8891503f9349 -i td/test-erasure-code/9/new.json 2026-02-19T16:20:42.238 INFO:tasks.workunit.client.0.trial127.stdout:9 2026-02-19T16:20:42.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-19T16:20:42.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBaOJdpao9+ABAAx391DZQOwR4ViaqlQbI7rg== --osd-uuid 90109f52-d38c-40fb-b7d1-8891503f9349 2026-02-19T16:20:42.261 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.259+0000 7f6382e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:42.263 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.260+0000 7f6382e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:42.264 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.261+0000 7f6382e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:42.264 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.261+0000 7f6382e37780 -1 bdev(0x3094c00 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:42.264 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.261+0000 7f6382e37780 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-19T16:20:42.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-19T16:20:42.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:42.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-19T16:20:42.600 INFO:tasks.workunit.client.0.trial127.stdout:adding osd9 key to auth repository 2026-02-19T16:20:42.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:42.919 INFO:tasks.workunit.client.0.trial127.stdout:start osd.9 2026-02-19T16:20:42.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-19T16:20:42.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:42.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:42.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:42.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:42.935 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.932+0000 7f2ca5a3e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:42.936 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.933+0000 7f2ca5a3e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:42.937 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:42.934+0000 7f2ca5a3e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:43.116 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:43.113+0000 7f2ca5a3e780 -1 Falling back to public interface 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:43.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T16:20:43.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:43.463 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:43.460+0000 7f2ca5a3e780 -1 osd.9 0 log_to_monitors true 2026-02-19T16:20:44.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:44.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:44.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:44.370 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:44.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:44.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T16:20:44.597 INFO:tasks.workunit.client.0.trial127.stdout:osd.9 up in weight 1 up_from 47 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/485785150,v1:127.0.0.1:6875/485785150] [v2:127.0.0.1:6876/485785150,v1:127.0.0.1:6877/485785150] exists,up 90109f52-d38c-40fb-b7d1-8891503f9349 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:20:44.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:20:44.602 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:20:44.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:20:44.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:20:44.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:20:44.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:20:44.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:20:44.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-19T16:20:44.604 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:20:44.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=57e270de-6850-404d-a968-ba97736346d9 2026-02-19T16:20:44.605 INFO:tasks.workunit.client.0.trial127.stdout:add osd10 57e270de-6850-404d-a968-ba97736346d9 2026-02-19T16:20:44.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 57e270de-6850-404d-a968-ba97736346d9' 2026-02-19T16:20:44.605 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:20:44.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBcOJdpVYRaJBAAAqw1nIFaMG+tDglBVzNZng== 2026-02-19T16:20:44.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBcOJdpVYRaJBAAAqw1nIFaMG+tDglBVzNZng=="}' 2026-02-19T16:20:44.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 57e270de-6850-404d-a968-ba97736346d9 -i td/test-erasure-code/10/new.json 2026-02-19T16:20:44.840 INFO:tasks.workunit.client.0.trial127.stdout:10 2026-02-19T16:20:44.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-19T16:20:44.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBcOJdpVYRaJBAAAqw1nIFaMG+tDglBVzNZng== --osd-uuid 57e270de-6850-404d-a968-ba97736346d9 2026-02-19T16:20:44.864 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:44.861+0000 7f05b8fd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:44.865 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:44.862+0000 7f05b8fd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:44.866 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:44.863+0000 7f05b8fd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:44.866 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:44.863+0000 7f05b8fd4780 -1 bdev(0x34f4c00 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-19T16:20:44.866 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:44.863+0000 7f05b8fd4780 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-19T16:20:45.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-19T16:20:45.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:20:45.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-19T16:20:45.185 INFO:tasks.workunit.client.0.trial127.stdout:adding osd10 key to auth repository 2026-02-19T16:20:45.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:20:45.504 INFO:tasks.workunit.client.0.trial127.stdout:start osd.10 2026-02-19T16:20:45.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-19T16:20:45.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:20:45.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:20:45.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:20:45.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:20:45.519 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:45.516+0000 7f93fb43a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:45.520 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:45.517+0000 7f93fb43a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:45.521 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:45.518+0000 7f93fb43a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:20:45.667 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:45.664+0000 7f93fb43a780 -1 Falling back to public interface 2026-02-19T16:20:45.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-19T16:20:45.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:20:45.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-19T16:20:45.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:20:45.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:20:45.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:45.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:20:45.731 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:20:45.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:45.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-19T16:20:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:20:45.985 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:20:45.982+0000 7f93fb43a780 -1 osd.10 0 log_to_monitors true 2026-02-19T16:20:46.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:20:46.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:20:46.959 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:20:46.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:20:46.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:20:46.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-19T16:20:47.186 INFO:tasks.workunit.client.0.trial127.stdout:osd.10 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/3940691999,v1:127.0.0.1:6883/3940691999] [v2:127.0.0.1:6884/3940691999,v1:127.0.0.1:6885/3940691999] exists,up 57e270de-6850-404d-a968-ba97736346d9 2026-02-19T16:20:47.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:20:47.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:20:47.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:20:47.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-19T16:20:47.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T16:20:47.404 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' does not exist 2026-02-19T16:20:47.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T16:20:47.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:20:47.694 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:20:47.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:20:48.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T16:20:48.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-19T16:20:48.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:20:48.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:20:48.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:20:48.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:20:48.938 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:20:48.938 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:20:48.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:20:48.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:20:48.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:20:49.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:20:49.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:20:49.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:20:49.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:20:49.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:20:49.020 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:20:49.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:20:49.248 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.249 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:20:49.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-19T16:20:49.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-19T16:20:49.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488' 2026-02-19T16:20:49.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:20:49.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672968 2026-02-19T16:20:49.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672968 2026-02-19T16:20:49.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968' 2026-02-19T16:20:49.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:20:49.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542151 2026-02-19T16:20:49.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542151 2026-02-19T16:20:49.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151' 2026-02-19T16:20:49.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.441 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:20:49.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-19T16:20:49.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-19T16:20:49.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335' 2026-02-19T16:20:49.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.505 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:20:49.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247814 2026-02-19T16:20:49.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247814 2026-02-19T16:20:49.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335 4-98784247814' 2026-02-19T16:20:49.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.570 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:20:49.635 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084294 2026-02-19T16:20:49.635 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084294 2026-02-19T16:20:49.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335 4-98784247814 5-120259084294' 2026-02-19T16:20:49.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.636 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:20:49.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920774 2026-02-19T16:20:49.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920774 2026-02-19T16:20:49.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335 4-98784247814 5-120259084294 6-141733920774' 2026-02-19T16:20:49.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:20:49.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757253 2026-02-19T16:20:49.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757253 2026-02-19T16:20:49.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335 4-98784247814 5-120259084294 6-141733920774 7-163208757253' 2026-02-19T16:20:49.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.765 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:20:49.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-19T16:20:49.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-19T16:20:49.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335 4-98784247814 5-120259084294 6-141733920774 7-163208757253 8-184683593732' 2026-02-19T16:20:49.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:20:49.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462916 2026-02-19T16:20:49.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462916 2026-02-19T16:20:49.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335 4-98784247814 5-120259084294 6-141733920774 7-163208757253 8-184683593732 9-201863462916' 2026-02-19T16:20:49.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:49.895 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:20:49.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299395 2026-02-19T16:20:49.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299395 2026-02-19T16:20:49.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-77309411335 4-98784247814 5-120259084294 6-141733920774 7-163208757253 8-184683593732 9-201863462916 10-223338299395' 2026-02-19T16:20:49.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:49.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836488 2026-02-19T16:20:49.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:49.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:20:49.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-19T16:20:49.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:49.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836488 2026-02-19T16:20:49.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-19T16:20:49.960 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 21474836488 2026-02-19T16:20:49.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:20:50.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-19T16:20:50.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:50.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672968 2026-02-19T16:20:50.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:50.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:20:50.191 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672968 2026-02-19T16:20:50.191 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:50.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672968 2026-02-19T16:20:50.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-19T16:20:50.193 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 42949672968 2026-02-19T16:20:50.193 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:20:50.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672966 -lt 42949672968 2026-02-19T16:20:50.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:20:51.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:20:51.423 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:20:51.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-19T16:20:51.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:51.647 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542151 2026-02-19T16:20:51.647 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:51.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:20:51.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542151 2026-02-19T16:20:51.649 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:51.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542151 2026-02-19T16:20:51.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-19T16:20:51.650 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 60129542151 2026-02-19T16:20:51.650 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:20:51.878 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542151 -lt 60129542151 2026-02-19T16:20:51.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:51.879 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411335 2026-02-19T16:20:51.880 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:51.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:20:51.881 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411335 2026-02-19T16:20:51.881 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:51.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411335 2026-02-19T16:20:51.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411335' 2026-02-19T16:20:51.882 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 77309411335 2026-02-19T16:20:51.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:20:52.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-19T16:20:52.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:52.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247814 2026-02-19T16:20:52.106 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:52.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:20:52.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247814 2026-02-19T16:20:52.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:52.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247814 2026-02-19T16:20:52.108 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 98784247814 2026-02-19T16:20:52.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247814' 2026-02-19T16:20:52.109 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:20:52.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247814 -lt 98784247814 2026-02-19T16:20:52.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:52.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-120259084294 2026-02-19T16:20:52.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:52.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:20:52.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084294 2026-02-19T16:20:52.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:52.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=120259084294 2026-02-19T16:20:52.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 120259084294' 2026-02-19T16:20:52.335 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 120259084294 2026-02-19T16:20:52.335 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:20:52.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 120259084294 -lt 120259084294 2026-02-19T16:20:52.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:52.558 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920774 2026-02-19T16:20:52.558 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:52.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:20:52.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920774 2026-02-19T16:20:52.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:52.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920774 2026-02-19T16:20:52.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920774' 2026-02-19T16:20:52.561 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 141733920774 2026-02-19T16:20:52.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:20:52.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-19T16:20:52.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:52.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757253 2026-02-19T16:20:52.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:52.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:20:52.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757253 2026-02-19T16:20:52.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:52.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757253 2026-02-19T16:20:52.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757253' 2026-02-19T16:20:52.792 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.7 seq 163208757253 2026-02-19T16:20:52.792 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:20:53.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757253 -lt 163208757253 2026-02-19T16:20:53.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:53.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593732 2026-02-19T16:20:53.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:53.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:20:53.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593732 2026-02-19T16:20:53.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:53.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593732 2026-02-19T16:20:53.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593732' 2026-02-19T16:20:53.019 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.8 seq 184683593732 2026-02-19T16:20:53.019 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:20:53.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-19T16:20:53.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:53.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462916 2026-02-19T16:20:53.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:53.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:20:53.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462916 2026-02-19T16:20:53.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:53.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462916 2026-02-19T16:20:53.245 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.9 seq 201863462916 2026-02-19T16:20:53.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462916' 2026-02-19T16:20:53.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:20:53.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462916 -lt 201863462916 2026-02-19T16:20:53.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:53.475 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299395 2026-02-19T16:20:53.475 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:53.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:20:53.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299395 2026-02-19T16:20:53.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:53.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299395 2026-02-19T16:20:53.478 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.10 seq 223338299395 2026-02-19T16:20:53.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299395' 2026-02-19T16:20:53.478 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:20:53.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299396 -lt 223338299395 2026-02-19T16:20:53.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:20:53.706 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:20:53.706 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:20:54.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T16:20:54.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:20:54.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:20:54.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:20:54.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:20:54.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:20:54.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:20:54.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:20:54.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T16:20:54.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:20:54.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:20:54.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:20:54.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T16:20:54.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:20:54.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:20:54.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-19T16:20:54.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:20:54.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:20:54.561 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:20:54.561 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:20:54.561 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:20:54.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.67892/ceph-osd.0.asok 2026-02-19T16:20:54.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-19T16:20:54.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.67892/ceph-osd.0.asok log flush 2026-02-19T16:20:54.606 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:20:54.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-19T16:20:54.611 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:20:20.226+0000 7f1d4b244780 0 load: jerasure load: lrc 2026-02-19T16:20:54.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-19T16:20:54.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-19T16:20:54.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-19T16:20:54.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-19T16:20:54.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-19T16:20:55.272 INFO:tasks.workunit.client.0.trial127.stderr:pool 'ecpool' already exists 2026-02-19T16:20:55.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:20:56.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:20:56.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:20:56.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:20:56.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:20:56.283 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:20:56.285 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:20:56.285 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:20:56.285 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:20:56.285 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:20:56.285 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:20:56.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:20:56.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:20:56.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:20:56.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:20:56.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:20:56.369 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:20:56.594 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:20:56.595 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:20:56.595 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:20:56.595 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:20:56.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:20:56.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:56.595 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:20:56.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836492 2026-02-19T16:20:56.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836492 2026-02-19T16:20:56.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492' 2026-02-19T16:20:56.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:56.662 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:20:56.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672971 2026-02-19T16:20:56.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672971 2026-02-19T16:20:56.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971' 2026-02-19T16:20:56.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:56.729 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:20:56.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542155 2026-02-19T16:20:56.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542155 2026-02-19T16:20:56.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155' 2026-02-19T16:20:56.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:56.793 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:20:56.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411338 2026-02-19T16:20:56.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411338 2026-02-19T16:20:56.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338' 2026-02-19T16:20:56.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:56.860 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:20:56.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247818 2026-02-19T16:20:56.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247818 2026-02-19T16:20:56.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338 4-98784247818' 2026-02-19T16:20:56.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:56.924 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:20:56.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084297 2026-02-19T16:20:56.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084297 2026-02-19T16:20:56.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338 4-98784247818 5-120259084297' 2026-02-19T16:20:56.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:56.988 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:20:57.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920778 2026-02-19T16:20:57.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920778 2026-02-19T16:20:57.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338 4-98784247818 5-120259084297 6-141733920778' 2026-02-19T16:20:57.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:57.051 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:20:57.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757256 2026-02-19T16:20:57.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757256 2026-02-19T16:20:57.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338 4-98784247818 5-120259084297 6-141733920778 7-163208757256' 2026-02-19T16:20:57.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:57.115 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:20:57.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593736 2026-02-19T16:20:57.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593736 2026-02-19T16:20:57.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338 4-98784247818 5-120259084297 6-141733920778 7-163208757256 8-184683593736' 2026-02-19T16:20:57.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:57.183 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:20:57.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462919 2026-02-19T16:20:57.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462919 2026-02-19T16:20:57.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338 4-98784247818 5-120259084297 6-141733920778 7-163208757256 8-184683593736 9-201863462919' 2026-02-19T16:20:57.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:20:57.247 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:20:57.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299399 2026-02-19T16:20:57.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299399 2026-02-19T16:20:57.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-77309411338 4-98784247818 5-120259084297 6-141733920778 7-163208757256 8-184683593736 9-201863462919 10-223338299399' 2026-02-19T16:20:57.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:57.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836492 2026-02-19T16:20:57.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:57.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:20:57.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836492 2026-02-19T16:20:57.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:57.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836492 2026-02-19T16:20:57.314 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 21474836492 2026-02-19T16:20:57.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-19T16:20:57.315 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:20:57.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-19T16:20:57.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:57.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672971 2026-02-19T16:20:57.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:57.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:20:57.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672971 2026-02-19T16:20:57.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:57.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672971 2026-02-19T16:20:57.547 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 42949672971 2026-02-19T16:20:57.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-19T16:20:57.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:20:57.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-19T16:20:57.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:57.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542155 2026-02-19T16:20:57.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:57.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:20:57.780 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542155 2026-02-19T16:20:57.780 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:57.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542155 2026-02-19T16:20:57.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-19T16:20:57.781 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 60129542155 2026-02-19T16:20:57.782 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:20:58.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-19T16:20:58.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:58.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411338 2026-02-19T16:20:58.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:58.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:20:58.012 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411338 2026-02-19T16:20:58.012 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:58.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411338 2026-02-19T16:20:58.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411338' 2026-02-19T16:20:58.014 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 77309411338 2026-02-19T16:20:58.014 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:20:58.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-19T16:20:58.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:58.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247818 2026-02-19T16:20:58.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:58.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:20:58.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247818 2026-02-19T16:20:58.240 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:58.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247818 2026-02-19T16:20:58.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247818' 2026-02-19T16:20:58.241 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 98784247818 2026-02-19T16:20:58.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:20:58.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247818 -lt 98784247818 2026-02-19T16:20:58.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:58.465 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-120259084297 2026-02-19T16:20:58.465 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:58.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:20:58.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084297 2026-02-19T16:20:58.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:58.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=120259084297 2026-02-19T16:20:58.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 120259084297' 2026-02-19T16:20:58.468 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 120259084297 2026-02-19T16:20:58.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:20:58.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 120259084297 -lt 120259084297 2026-02-19T16:20:58.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:58.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920778 2026-02-19T16:20:58.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:58.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:20:58.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920778 2026-02-19T16:20:58.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:58.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920778 2026-02-19T16:20:58.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920778' 2026-02-19T16:20:58.699 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 141733920778 2026-02-19T16:20:58.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:20:58.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920778 -lt 141733920778 2026-02-19T16:20:58.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:58.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757256 2026-02-19T16:20:58.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:58.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:20:58.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757256 2026-02-19T16:20:58.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757256 2026-02-19T16:20:58.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757256' 2026-02-19T16:20:58.929 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.7 seq 163208757256 2026-02-19T16:20:58.929 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:20:59.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757256 -lt 163208757256 2026-02-19T16:20:59.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:59.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593736 2026-02-19T16:20:59.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:59.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:20:59.161 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593736 2026-02-19T16:20:59.161 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:59.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593736 2026-02-19T16:20:59.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593736' 2026-02-19T16:20:59.163 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.8 seq 184683593736 2026-02-19T16:20:59.163 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:20:59.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593736 -lt 184683593736 2026-02-19T16:20:59.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:59.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462919 2026-02-19T16:20:59.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:59.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:20:59.387 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462919 2026-02-19T16:20:59.387 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:59.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462919 2026-02-19T16:20:59.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462919' 2026-02-19T16:20:59.389 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.9 seq 201863462919 2026-02-19T16:20:59.389 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:20:59.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-19T16:20:59.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:20:59.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299399 2026-02-19T16:20:59.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:20:59.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:20:59.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299399 2026-02-19T16:20:59.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:20:59.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299399 2026-02-19T16:20:59.618 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299399' 2026-02-19T16:20:59.618 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.10 seq 223338299399 2026-02-19T16:20:59.618 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:20:59.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299399 -lt 223338299399 2026-02-19T16:20:59.843 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:20:59.844 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:20:59.844 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:00.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-19T16:21:00.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:21:00.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:21:00.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:21:00.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:21:00.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:21:00.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:21:00.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:21:00.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-19T16:21:00.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:21:00.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:00.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:00.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-19T16:21:00.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:21:00.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:21:00.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-19T16:21:00.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T16:21:00.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-19T16:21:00.700 INFO:tasks.workunit.client.0.trial127.stderr:TEST_chunk_mapping 2026-02-19T16:21:00.700 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_put_get_isa 2026-02-19T16:21:00.700 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_put_get_jerasure 2026-02-19T16:21:00.700 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_put_get_lrc_advanced 2026-02-19T16:21:00.700 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_put_get_lrc_kml 2026-02-19T16:21:00.700 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_put_get_shec' 2026-02-19T16:21:00.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T16:21:00.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-19T16:21:00.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: local payload=ABC 2026-02-19T16:21:00.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:267: TEST_alignment_constraints: echo ABC 2026-02-19T16:21:00.701 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: chunk_size 2026-02-19T16:21:00.701 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T16:21:00.709 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-19T16:21:00.709 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-19T16:21:00.709 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-19T16:21:00.718 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-19T16:21:00.718 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-19T16:21:00.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-19T16:21:00.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-19T16:21:00.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: local stripe_unit=4096 2026-02-19T16:21:00.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-19T16:21:00.720 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: grep k= 2026-02-19T16:21:00.952 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: eval local k=2 2026-02-19T16:21:00.953 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: local k=2 2026-02-19T16:21:00.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:275: TEST_alignment_constraints: local block_size=8191 2026-02-19T16:21:00.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:276: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-19T16:21:00.953 INFO:tasks.workunit.client.0.trial127.stderr:2+0 records in 2026-02-19T16:21:00.954 INFO:tasks.workunit.client.0.trial127.stderr:2+0 records out 2026-02-19T16:21:00.954 INFO:tasks.workunit.client.0.trial127.stderr:16382 bytes (16 kB, 16 KiB) copied, 6.3368e-05 s, 259 MB/s 2026-02-19T16:21:00.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:277: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-19T16:21:00.968 INFO:tasks.workunit.client.0.trial127.stderr:INFO: op_size has been rounded to 8192 2026-02-19T16:21:00.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:279: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:21:00.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:320: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=ecpool 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=0 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=1 2026-02-19T16:21:00.974 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-19T16:21:00.975 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T16:21:00.983 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-19T16:21:00.983 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-19T16:21:00.984 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-19T16:21:00.992 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-19T16:21:00.992 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-19T16:21:00.992 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-19T16:21:00.993 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-19T16:21:00.993 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-19T16:21:00.993 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-19T16:21:00.993 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T16:21:01.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-19T16:21:01.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-19T16:21:01.001 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-19T16:21:01.010 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-19T16:21:01.010 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-19T16:21:01.010 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-19T16:21:01.010 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-19T16:21:01.010 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-19T16:21:01.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-19T16:21:01.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-19T16:21:01.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-19T16:21:01.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-19T16:21:01.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-19T16:21:01.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T16:21:01.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-19T16:21:01.046 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-19T16:21:01.046 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:01.271 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:21:01.271 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr:0' 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-19T16:21:01.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-19T16:21:01.323 INFO:tasks.workunit.client.0.trial127.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T16:21:01.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-19T16:21:01.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-19T16:21:01.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-19T16:21:01.377 INFO:tasks.workunit.client.0.trial127.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T16:21:01.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-19T16:21:01.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-19T16:21:01.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-19T16:21:01.428 INFO:tasks.workunit.client.0.trial127.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T16:21:01.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-19T16:21:01.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-19T16:21:01.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-19T16:21:01.480 INFO:tasks.workunit.client.0.trial127.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T16:21:01.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-19T16:21:01.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-19T16:21:01.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:01.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-19T16:21:01.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-19T16:21:01.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T16:21:01.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-19T16:21:01.484 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-19T16:21:01.485 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:01.711 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:21:01.711 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:21:01.711 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:0' 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-19T16:21:01.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:21:01.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-19T16:21:02.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-19T16:21:02.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-19T16:21:02.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:21:02.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:21:02.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:start osd.3 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:21:02.116 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-19T16:21:02.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T16:21:02.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:21:02.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:21:02.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:21:02.127 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:02.125+0000 7f551a448780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:02.128 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:02.127+0000 7f551a448780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:02.129 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:02.128+0000 7f551a448780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:02.256 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:02.254+0000 7f551a448780 -1 Falling back to public interface 2026-02-19T16:21:02.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T16:21:02.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:21:02.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:21:02.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:21:02.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:21:02.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:02.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:21:02.344 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:02.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:02.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:21:02.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:21:02.657 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:02.655+0000 7f551a448780 -1 osd.3 70 log_to_monitors true 2026-02-19T16:21:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:21:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:21:03.574 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:osd.3 up in weight 1 up_from 73 up_thru 73 down_at 70 last_clean_interval [18,70) [v2:127.0.0.1:6826/771440565,v1:127.0.0.1:6827/771440565] [v2:127.0.0.1:6828/771440565,v1:127.0.0.1:6829/771440565] exists,up e5db492a-74d3-4383-8045-6600e28aab52 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:21:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:21:03.800 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:21:03.800 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:21:03.800 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:21:03.800 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:21:03.800 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:21:03.800 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:21:03.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:21:03.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:21:03.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:21:03.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:21:03.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:21:03.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:21:04.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:21:04.110 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:04.110 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:21:04.110 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:21:04.110 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:21:04.110 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.111 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:21:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836495 2026-02-19T16:21:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836495 2026-02-19T16:21:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495' 2026-02-19T16:21:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.175 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:21:04.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672975 2026-02-19T16:21:04.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672975 2026-02-19T16:21:04.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975' 2026-02-19T16:21:04.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:21:04.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542158 2026-02-19T16:21:04.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542158 2026-02-19T16:21:04.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158' 2026-02-19T16:21:04.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.304 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:21:04.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612611 2026-02-19T16:21:04.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612611 2026-02-19T16:21:04.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611' 2026-02-19T16:21:04.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.367 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:21:04.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247821 2026-02-19T16:21:04.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247821 2026-02-19T16:21:04.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611 4-98784247821' 2026-02-19T16:21:04.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.433 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:21:04.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084301 2026-02-19T16:21:04.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084301 2026-02-19T16:21:04.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611 4-98784247821 5-120259084301' 2026-02-19T16:21:04.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:21:04.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920781 2026-02-19T16:21:04.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920781 2026-02-19T16:21:04.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611 4-98784247821 5-120259084301 6-141733920781' 2026-02-19T16:21:04.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:21:04.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757260 2026-02-19T16:21:04.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757260 2026-02-19T16:21:04.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611 4-98784247821 5-120259084301 6-141733920781 7-163208757260' 2026-02-19T16:21:04.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.629 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:21:04.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593739 2026-02-19T16:21:04.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593739 2026-02-19T16:21:04.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611 4-98784247821 5-120259084301 6-141733920781 7-163208757260 8-184683593739' 2026-02-19T16:21:04.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.692 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:21:04.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462923 2026-02-19T16:21:04.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462923 2026-02-19T16:21:04.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611 4-98784247821 5-120259084301 6-141733920781 7-163208757260 8-184683593739 9-201863462923' 2026-02-19T16:21:04.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:04.756 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:21:04.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299402 2026-02-19T16:21:04.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299402 2026-02-19T16:21:04.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-313532612611 4-98784247821 5-120259084301 6-141733920781 7-163208757260 8-184683593739 9-201863462923 10-223338299402' 2026-02-19T16:21:04.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:04.821 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836495 2026-02-19T16:21:04.821 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:04.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:21:04.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836495 2026-02-19T16:21:04.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:04.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836495 2026-02-19T16:21:04.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-19T16:21:04.823 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 21474836495 2026-02-19T16:21:04.823 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:05.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-19T16:21:05.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:21:06.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:21:06.046 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:06.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-19T16:21:06.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:06.273 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672975 2026-02-19T16:21:06.273 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:06.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:21:06.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672975 2026-02-19T16:21:06.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:06.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672975 2026-02-19T16:21:06.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2026-02-19T16:21:06.275 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 42949672975 2026-02-19T16:21:06.275 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:21:06.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672975 -lt 42949672975 2026-02-19T16:21:06.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:06.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542158 2026-02-19T16:21:06.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:06.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:21:06.500 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542158 2026-02-19T16:21:06.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:06.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542158 2026-02-19T16:21:06.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-19T16:21:06.502 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 60129542158 2026-02-19T16:21:06.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:21:06.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-19T16:21:06.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:06.729 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-313532612611 2026-02-19T16:21:06.729 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:06.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:21:06.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612611 2026-02-19T16:21:06.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:06.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=313532612611 2026-02-19T16:21:06.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 313532612611' 2026-02-19T16:21:06.732 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 313532612611 2026-02-19T16:21:06.732 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:21:06.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 313532612611 -lt 313532612611 2026-02-19T16:21:06.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:06.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247821 2026-02-19T16:21:06.962 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:06.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:21:06.963 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247821 2026-02-19T16:21:06.963 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:06.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247821 2026-02-19T16:21:06.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247821' 2026-02-19T16:21:06.964 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.4 seq 98784247821 2026-02-19T16:21:06.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:21:07.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247821 -lt 98784247821 2026-02-19T16:21:07.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:07.185 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-120259084301 2026-02-19T16:21:07.185 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:07.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:21:07.186 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084301 2026-02-19T16:21:07.186 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:07.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=120259084301 2026-02-19T16:21:07.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 120259084301' 2026-02-19T16:21:07.187 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.5 seq 120259084301 2026-02-19T16:21:07.187 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:21:07.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 120259084301 -lt 120259084301 2026-02-19T16:21:07.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:07.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920781 2026-02-19T16:21:07.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:07.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:21:07.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920781 2026-02-19T16:21:07.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:07.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920781 2026-02-19T16:21:07.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920781' 2026-02-19T16:21:07.418 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.6 seq 141733920781 2026-02-19T16:21:07.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:21:07.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920782 -lt 141733920781 2026-02-19T16:21:07.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:07.645 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757260 2026-02-19T16:21:07.645 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:07.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:21:07.647 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757260 2026-02-19T16:21:07.647 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:07.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757260 2026-02-19T16:21:07.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757260' 2026-02-19T16:21:07.648 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.7 seq 163208757260 2026-02-19T16:21:07.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:21:07.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757260 -lt 163208757260 2026-02-19T16:21:07.877 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:07.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593739 2026-02-19T16:21:07.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:07.878 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:21:07.878 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593739 2026-02-19T16:21:07.878 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:07.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593739 2026-02-19T16:21:07.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593739' 2026-02-19T16:21:07.880 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.8 seq 184683593739 2026-02-19T16:21:07.880 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:21:08.111 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593740 -lt 184683593739 2026-02-19T16:21:08.111 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:08.111 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462923 2026-02-19T16:21:08.111 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:08.112 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:21:08.113 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462923 2026-02-19T16:21:08.113 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:08.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462923 2026-02-19T16:21:08.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462923' 2026-02-19T16:21:08.114 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.9 seq 201863462923 2026-02-19T16:21:08.114 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:21:08.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462923 -lt 201863462923 2026-02-19T16:21:08.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:08.336 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299402 2026-02-19T16:21:08.337 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:08.337 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:21:08.338 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299402 2026-02-19T16:21:08.338 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:08.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299402 2026-02-19T16:21:08.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299402' 2026-02-19T16:21:08.339 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.10 seq 223338299402 2026-02-19T16:21:08.339 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:21:08.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299403 -lt 223338299402 2026-02-19T16:21:08.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:21:08.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:08.564 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:21:08.884 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:21:09.111 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-19T16:21:09.111 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:21:09.111 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:09.112 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:09.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-19T16:21:09.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:21:09.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:21:09.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-19T16:21:09.432 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:21:09.433 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:21:09.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:21:09.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:21:09.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:21:09.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:21:09.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-19T16:21:09.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T16:21:09.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:21:09.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T16:21:09.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:21:09.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-19T16:21:09.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-19T16:21:09.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-19T16:21:09.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T16:21:09.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:21:09.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:09.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:start osd.5 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:21:09.837 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-19T16:21:09.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T16:21:09.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:21:09.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:21:09.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:21:09.851 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:09.850+0000 7f6fad7dd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:09.853 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:09.851+0000 7f6fad7dd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:09.854 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:09.852+0000 7f6fad7dd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:10.002 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:10.000+0000 7f6fad7dd780 -1 Falling back to public interface 2026-02-19T16:21:10.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T16:21:10.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:21:10.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T16:21:10.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:21:10.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:21:10.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:10.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:21:10.065 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:10.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:10.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:21:10.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:21:10.394 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:10.392+0000 7f6fad7dd780 -1 osd.5 75 log_to_monitors true 2026-02-19T16:21:11.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:21:11.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:11.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:21:11.290 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:11.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:11.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:21:11.525 INFO:tasks.workunit.client.0.trial127.stderr:osd.5 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [28,75) [v2:127.0.0.1:6842/2743020773,v1:127.0.0.1:6843/2743020773] [v2:127.0.0.1:6844/2743020773,v1:127.0.0.1:6845/2743020773] exists,up d4538fb7-dda6-45d2-9e55-e3cb1da6a649 2026-02-19T16:21:11.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:21:11.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:21:11.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:21:11.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:21:11.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:21:11.526 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:21:11.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:21:11.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:21:11.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:21:11.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:21:11.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:21:11.607 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:21:11.834 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:21:11.835 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:11.836 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:21:11.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:21:11.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:11.836 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:21:11.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-19T16:21:11.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-19T16:21:11.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499' 2026-02-19T16:21:11.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:11.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:21:11.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672978 2026-02-19T16:21:11.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672978 2026-02-19T16:21:11.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978' 2026-02-19T16:21:11.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:11.970 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:21:12.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542162 2026-02-19T16:21:12.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542162 2026-02-19T16:21:12.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162' 2026-02-19T16:21:12.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:21:12.099 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612614 2026-02-19T16:21:12.099 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612614 2026-02-19T16:21:12.099 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614' 2026-02-19T16:21:12.099 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.099 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:21:12.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247825 2026-02-19T16:21:12.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247825 2026-02-19T16:21:12.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614 4-98784247825' 2026-02-19T16:21:12.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.163 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:21:12.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449091 2026-02-19T16:21:12.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449091 2026-02-19T16:21:12.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614 4-98784247825 5-335007449091' 2026-02-19T16:21:12.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.226 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:21:12.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920785 2026-02-19T16:21:12.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920785 2026-02-19T16:21:12.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614 4-98784247825 5-335007449091 6-141733920785' 2026-02-19T16:21:12.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.290 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:21:12.357 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757263 2026-02-19T16:21:12.357 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757263 2026-02-19T16:21:12.357 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614 4-98784247825 5-335007449091 6-141733920785 7-163208757263' 2026-02-19T16:21:12.357 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.357 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:21:12.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593743 2026-02-19T16:21:12.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593743 2026-02-19T16:21:12.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614 4-98784247825 5-335007449091 6-141733920785 7-163208757263 8-184683593743' 2026-02-19T16:21:12.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.420 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:21:12.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462926 2026-02-19T16:21:12.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462926 2026-02-19T16:21:12.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614 4-98784247825 5-335007449091 6-141733920785 7-163208757263 8-184683593743 9-201863462926' 2026-02-19T16:21:12.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:12.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:21:12.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299406 2026-02-19T16:21:12.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299406 2026-02-19T16:21:12.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-313532612614 4-98784247825 5-335007449091 6-141733920785 7-163208757263 8-184683593743 9-201863462926 10-223338299406' 2026-02-19T16:21:12.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:12.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836499 2026-02-19T16:21:12.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:12.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:21:12.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-19T16:21:12.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:12.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836499 2026-02-19T16:21:12.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-19T16:21:12.551 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 21474836499 2026-02-19T16:21:12.551 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:12.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836496 -lt 21474836499 2026-02-19T16:21:12.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:21:13.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:21:13.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:14.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-19T16:21:14.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:14.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672978 2026-02-19T16:21:14.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:14.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:21:14.006 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672978 2026-02-19T16:21:14.007 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:14.008 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672978 2026-02-19T16:21:14.008 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672978' 2026-02-19T16:21:14.008 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 42949672978 2026-02-19T16:21:14.008 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:21:14.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672978 -lt 42949672978 2026-02-19T16:21:14.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:14.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542162 2026-02-19T16:21:14.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:14.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:21:14.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542162 2026-02-19T16:21:14.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:14.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542162 2026-02-19T16:21:14.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542162' 2026-02-19T16:21:14.236 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 60129542162 2026-02-19T16:21:14.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:21:14.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542162 -lt 60129542162 2026-02-19T16:21:14.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:14.463 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-313532612614 2026-02-19T16:21:14.463 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:14.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:21:14.464 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612614 2026-02-19T16:21:14.465 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:14.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=313532612614 2026-02-19T16:21:14.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 313532612614' 2026-02-19T16:21:14.466 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 313532612614 2026-02-19T16:21:14.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:21:14.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 313532612615 -lt 313532612614 2026-02-19T16:21:14.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:14.693 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247825 2026-02-19T16:21:14.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:14.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:21:14.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247825 2026-02-19T16:21:14.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:14.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247825 2026-02-19T16:21:14.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247825' 2026-02-19T16:21:14.697 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.4 seq 98784247825 2026-02-19T16:21:14.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:21:14.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247825 -lt 98784247825 2026-02-19T16:21:14.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:14.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-335007449091 2026-02-19T16:21:14.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:14.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:21:14.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-335007449091 2026-02-19T16:21:14.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:14.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449091 2026-02-19T16:21:14.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 335007449091' 2026-02-19T16:21:14.929 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.5 seq 335007449091 2026-02-19T16:21:14.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:21:15.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-19T16:21:15.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:15.156 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920785 2026-02-19T16:21:15.156 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:15.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:21:15.157 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920785 2026-02-19T16:21:15.157 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:15.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920785 2026-02-19T16:21:15.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920785' 2026-02-19T16:21:15.158 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.6 seq 141733920785 2026-02-19T16:21:15.159 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:21:15.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920785 -lt 141733920785 2026-02-19T16:21:15.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:15.382 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757263 2026-02-19T16:21:15.383 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:15.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:21:15.384 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757263 2026-02-19T16:21:15.384 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:15.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757263 2026-02-19T16:21:15.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757263' 2026-02-19T16:21:15.385 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.7 seq 163208757263 2026-02-19T16:21:15.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:21:15.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757264 -lt 163208757263 2026-02-19T16:21:15.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:15.613 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593743 2026-02-19T16:21:15.613 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:15.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:21:15.614 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593743 2026-02-19T16:21:15.614 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:15.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593743 2026-02-19T16:21:15.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593743' 2026-02-19T16:21:15.615 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.8 seq 184683593743 2026-02-19T16:21:15.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:21:15.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593743 -lt 184683593743 2026-02-19T16:21:15.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:15.843 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462926 2026-02-19T16:21:15.843 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:15.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:21:15.844 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462926 2026-02-19T16:21:15.844 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:15.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462926 2026-02-19T16:21:15.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462926' 2026-02-19T16:21:15.845 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.9 seq 201863462926 2026-02-19T16:21:15.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:21:16.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462927 -lt 201863462926 2026-02-19T16:21:16.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:16.077 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299406 2026-02-19T16:21:16.077 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:16.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:21:16.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299406 2026-02-19T16:21:16.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:16.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299406 2026-02-19T16:21:16.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299406' 2026-02-19T16:21:16.079 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.10 seq 223338299406 2026-02-19T16:21:16.080 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:21:16.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299406 -lt 223338299406 2026-02-19T16:21:16.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:21:16.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:16.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:16.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-19T16:21:16.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:21:16.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:21:16.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:21:16.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:21:16.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:21:16.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:21:16.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:21:16.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-19T16:21:16.851 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:21:16.852 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:16.852 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:17.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-19T16:21:17.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:21:17.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:21:17.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:322: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-19T16:21:17.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-19T16:21:17.688 INFO:tasks.workunit.client.0.trial127.stdout:crush-device-class= 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:crush-failure-domain=host 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:crush-num-failure-domains=0 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:crush-osds-per-failure-domain=0 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:crush-root=default 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:k=-1 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:l=-1 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:layers=[ [ "cDD", "" ] ] 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:m=-1 2026-02-19T16:21:17.689 INFO:tasks.workunit.client.0.trial127.stdout:mapping=_DD 2026-02-19T16:21:17.690 INFO:tasks.workunit.client.0.trial127.stdout:plugin=lrc 2026-02-19T16:21:17.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:328: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-19T16:21:17.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-19T16:21:18.042 INFO:tasks.workunit.client.0.trial127.stderr:pool 'remap-pool' already exists 2026-02-19T16:21:18.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:21:19.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:336: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-19T16:21:19.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-19T16:21:19.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=remap-pool 2026-02-19T16:21:19.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=1 2026-02-19T16:21:19.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=2 2026-02-19T16:21:19.055 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-19T16:21:19.055 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T16:21:19.066 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-19T16:21:19.066 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-19T16:21:19.066 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-19T16:21:19.076 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-19T16:21:19.076 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-19T16:21:19.076 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-19T16:21:19.076 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-19T16:21:19.077 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-19T16:21:19.077 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-19T16:21:19.077 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T16:21:19.086 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-19T16:21:19.087 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-19T16:21:19.087 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-19T16:21:19.096 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-19T16:21:19.096 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-19T16:21:19.096 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-19T16:21:19.096 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-19T16:21:19.096 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-19T16:21:19.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-19T16:21:19.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-19T16:21:19.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-19T16:21:19.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-19T16:21:19.131 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-19T16:21:19.131 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-19T16:21:19.132 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-19T16:21:19.132 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-19T16:21:19.132 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:19.366 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr:9' 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-19T16:21:19.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-19T16:21:19.417 INFO:tasks.workunit.client.0.trial127.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T16:21:19.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-19T16:21:19.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-19T16:21:19.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-19T16:21:19.471 INFO:tasks.workunit.client.0.trial127.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T16:21:19.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-19T16:21:19.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-19T16:21:19.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-19T16:21:19.525 INFO:tasks.workunit.client.0.trial127.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T16:21:19.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-19T16:21:19.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-19T16:21:19.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:19.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-19T16:21:19.529 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-19T16:21:19.529 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-19T16:21:19.529 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-19T16:21:19.529 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-19T16:21:19.529 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:19.761 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T16:21:19.761 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:21:19.761 INFO:tasks.workunit.client.0.trial127.stderr:9' 2026-02-19T16:21:19.761 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-19T16:21:19.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-19T16:21:19.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-19T16:21:19.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:21:19.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:21:20.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:21:20.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:20.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T16:21:20.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:21:20.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-19T16:21:20.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:21:20.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-19T16:21:20.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:20.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:21:20.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:20.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:start osd.6 2026-02-19T16:21:20.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:21:20.172 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-19T16:21:20.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-19T16:21:20.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:21:20.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:21:20.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:21:20.181 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:20.178+0000 7ff7ff469780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:20.182 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:20.179+0000 7ff7ff469780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:20.183 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:20.180+0000 7ff7ff469780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:20.318 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:20.315+0000 7ff7ff469780 -1 Falling back to public interface 2026-02-19T16:21:20.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:20.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:21:20.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:21:20.677 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:20.674+0000 7ff7ff469780 -1 osd.6 84 log_to_monitors true 2026-02-19T16:21:21.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:21:21.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:21.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:21:21.621 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:21.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:21.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:21:21.845 INFO:tasks.workunit.client.0.trial127.stderr:osd.6 up in weight 1 up_from 88 up_thru 88 down_at 85 last_clean_interval [33,84) [v2:127.0.0.1:6850/4044044178,v1:127.0.0.1:6851/4044044178] [v2:127.0.0.1:6852/4044044178,v1:127.0.0.1:6853/4044044178] exists,up 373975fc-f0a9-4f3b-a655-bd5efae5a0c3 2026-02-19T16:21:21.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:21:21.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:21:21.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:21:21.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:21:21.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:21:21.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:21:21.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:21:21.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:21:21.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:21:21.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:21:21.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:21:22.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:21:22.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:21:22.153 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:21:22.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836503 2026-02-19T16:21:22.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836503 2026-02-19T16:21:22.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503' 2026-02-19T16:21:22.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.216 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:21:22.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672982 2026-02-19T16:21:22.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672982 2026-02-19T16:21:22.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982' 2026-02-19T16:21:22.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.280 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:21:22.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542166 2026-02-19T16:21:22.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542166 2026-02-19T16:21:22.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166' 2026-02-19T16:21:22.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.346 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:21:22.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612618 2026-02-19T16:21:22.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612618 2026-02-19T16:21:22.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618' 2026-02-19T16:21:22.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:21:22.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247829 2026-02-19T16:21:22.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247829 2026-02-19T16:21:22.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618 4-98784247829' 2026-02-19T16:21:22.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:21:22.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449095 2026-02-19T16:21:22.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449095 2026-02-19T16:21:22.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618 4-98784247829 5-335007449095' 2026-02-19T16:21:22.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.539 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:21:22.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122051 2026-02-19T16:21:22.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122051 2026-02-19T16:21:22.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618 4-98784247829 5-335007449095 6-377957122051' 2026-02-19T16:21:22.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.604 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:21:22.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757267 2026-02-19T16:21:22.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757267 2026-02-19T16:21:22.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618 4-98784247829 5-335007449095 6-377957122051 7-163208757267' 2026-02-19T16:21:22.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.668 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:21:22.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593747 2026-02-19T16:21:22.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593747 2026-02-19T16:21:22.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618 4-98784247829 5-335007449095 6-377957122051 7-163208757267 8-184683593747' 2026-02-19T16:21:22.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.732 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:21:22.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462930 2026-02-19T16:21:22.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462930 2026-02-19T16:21:22.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618 4-98784247829 5-335007449095 6-377957122051 7-163208757267 8-184683593747 9-201863462930' 2026-02-19T16:21:22.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:22.797 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:21:22.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299410 2026-02-19T16:21:22.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299410 2026-02-19T16:21:22.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-313532612618 4-98784247829 5-335007449095 6-377957122051 7-163208757267 8-184683593747 9-201863462930 10-223338299410' 2026-02-19T16:21:22.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:22.862 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836503 2026-02-19T16:21:22.862 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:22.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:21:22.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836503 2026-02-19T16:21:22.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:22.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836503 2026-02-19T16:21:22.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-19T16:21:22.865 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 21474836503 2026-02-19T16:21:22.865 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:23.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836500 -lt 21474836503 2026-02-19T16:21:23.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:21:24.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:21:24.092 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:24.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836503 -lt 21474836503 2026-02-19T16:21:24.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:24.318 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672982 2026-02-19T16:21:24.318 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:24.319 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:21:24.319 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672982 2026-02-19T16:21:24.319 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:24.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672982 2026-02-19T16:21:24.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672982' 2026-02-19T16:21:24.321 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 42949672982 2026-02-19T16:21:24.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:21:24.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672982 -lt 42949672982 2026-02-19T16:21:24.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:24.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542166 2026-02-19T16:21:24.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:24.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:21:24.551 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542166 2026-02-19T16:21:24.551 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:24.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542166 2026-02-19T16:21:24.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542166' 2026-02-19T16:21:24.552 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 60129542166 2026-02-19T16:21:24.552 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:21:24.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542166 -lt 60129542166 2026-02-19T16:21:24.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:24.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-313532612618 2026-02-19T16:21:24.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:24.780 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:21:24.780 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612618 2026-02-19T16:21:24.780 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:24.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=313532612618 2026-02-19T16:21:24.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 313532612618' 2026-02-19T16:21:24.781 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 313532612618 2026-02-19T16:21:24.782 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:21:25.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 313532612619 -lt 313532612618 2026-02-19T16:21:25.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:25.008 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247829 2026-02-19T16:21:25.008 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:25.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:21:25.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247829 2026-02-19T16:21:25.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:25.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247829 2026-02-19T16:21:25.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247829' 2026-02-19T16:21:25.011 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.4 seq 98784247829 2026-02-19T16:21:25.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:21:25.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247829 -lt 98784247829 2026-02-19T16:21:25.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:25.236 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-335007449095 2026-02-19T16:21:25.236 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:25.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:21:25.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-335007449095 2026-02-19T16:21:25.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:25.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449095 2026-02-19T16:21:25.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 335007449095' 2026-02-19T16:21:25.239 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.5 seq 335007449095 2026-02-19T16:21:25.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:21:25.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449095 -lt 335007449095 2026-02-19T16:21:25.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:25.465 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-377957122051 2026-02-19T16:21:25.465 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:25.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:21:25.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-377957122051 2026-02-19T16:21:25.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:25.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=377957122051 2026-02-19T16:21:25.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 377957122051' 2026-02-19T16:21:25.468 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.6 seq 377957122051 2026-02-19T16:21:25.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:21:25.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 377957122051 -lt 377957122051 2026-02-19T16:21:25.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:25.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757267 2026-02-19T16:21:25.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:25.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:21:25.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757267 2026-02-19T16:21:25.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:25.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757267 2026-02-19T16:21:25.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757267' 2026-02-19T16:21:25.699 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.7 seq 163208757267 2026-02-19T16:21:25.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:21:25.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757268 -lt 163208757267 2026-02-19T16:21:25.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:25.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593747 2026-02-19T16:21:25.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:25.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:21:25.934 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593747 2026-02-19T16:21:25.935 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:25.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593747 2026-02-19T16:21:25.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593747' 2026-02-19T16:21:25.936 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.8 seq 184683593747 2026-02-19T16:21:25.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:21:26.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593747 -lt 184683593747 2026-02-19T16:21:26.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:26.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462930 2026-02-19T16:21:26.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:26.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:21:26.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462930 2026-02-19T16:21:26.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:26.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462930 2026-02-19T16:21:26.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462930' 2026-02-19T16:21:26.167 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.9 seq 201863462930 2026-02-19T16:21:26.167 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:21:26.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462931 -lt 201863462930 2026-02-19T16:21:26.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:26.394 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299410 2026-02-19T16:21:26.394 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:26.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:21:26.395 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299410 2026-02-19T16:21:26.395 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:26.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299410 2026-02-19T16:21:26.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299410' 2026-02-19T16:21:26.396 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.10 seq 223338299410 2026-02-19T16:21:26.397 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:21:26.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299410 -lt 223338299410 2026-02-19T16:21:26.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:21:26.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:26.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:21:26.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:21:27.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T16:21:27.168 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:21:27.168 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:27.168 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:27.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T16:21:27.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:21:27.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:21:27.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T16:21:27.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:21:27.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-19T16:21:27.486 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:21:27.486 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:21:27.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:21:27.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:21:27.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:21:27.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:21:27.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:27.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T16:21:27.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:21:27.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T16:21:27.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:21:27.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-19T16:21:27.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-19T16:21:27.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-19T16:21:27.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T16:21:27.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:21:27.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T16:21:27.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:21:27.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-19T16:21:27.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:21:27.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:21:27.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:21:27.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:21:27.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-19T16:21:27.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T16:21:27.701 INFO:tasks.workunit.client.0.trial127.stderr:start osd.9 2026-02-19T16:21:27.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=51123eff-d3fc-4545-bdc6-aa3f9bdcd3ce --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.67892/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:21:27.701 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-19T16:21:27.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T16:21:27.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:21:27.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:21:27.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:21:27.711 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:27.709+0000 7f7154c4e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:27.712 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:27.710+0000 7f7154c4e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:27.713 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:27.711+0000 7f7154c4e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:21:27.851 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:27.850+0000 7f7154c4e780 -1 Falling back to public interface 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:27.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T16:21:28.150 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:21:28.158 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:21:28.156+0000 7f7154c4e780 -1 osd.9 89 log_to_monitors true 2026-02-19T16:21:29.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:21:29.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:21:29.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:21:29.152 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:29.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:21:29.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T16:21:29.380 INFO:tasks.workunit.client.0.trial127.stderr:osd.9 up in weight 1 up_from 93 up_thru 93 down_at 90 last_clean_interval [47,89) [v2:127.0.0.1:6874/396289713,v1:127.0.0.1:6875/396289713] [v2:127.0.0.1:6876/396289713,v1:127.0.0.1:6877/396289713] exists,up 90109f52-d38c-40fb-b7d1-8891503f9349 2026-02-19T16:21:29.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:21:29.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:21:29.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:21:29.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:21:29.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:21:29.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:21:29.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:21:29.381 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:21:29.381 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:21:29.381 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:21:29.381 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:21:29.381 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:21:29.381 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:21:29.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:21:29.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:21:29.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:21:29.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:21:29.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:21:29.462 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:21:29.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:21:29.686 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:29.686 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:21:29.686 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:21:29.686 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:21:29.686 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:21:29.686 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:21:29.687 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:21:29.687 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:21:29.687 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:29.687 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:21:29.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:21:29.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:29.687 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:21:29.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836506 2026-02-19T16:21:29.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836506 2026-02-19T16:21:29.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506' 2026-02-19T16:21:29.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:29.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:21:29.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672986 2026-02-19T16:21:29.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672986 2026-02-19T16:21:29.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986' 2026-02-19T16:21:29.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:29.816 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:21:29.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542169 2026-02-19T16:21:29.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542169 2026-02-19T16:21:29.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169' 2026-02-19T16:21:29.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:29.881 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:21:29.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612622 2026-02-19T16:21:29.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612622 2026-02-19T16:21:29.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622' 2026-02-19T16:21:29.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:29.947 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:21:30.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247832 2026-02-19T16:21:30.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247832 2026-02-19T16:21:30.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622 4-98784247832' 2026-02-19T16:21:30.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:30.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:21:30.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449098 2026-02-19T16:21:30.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449098 2026-02-19T16:21:30.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622 4-98784247832 5-335007449098' 2026-02-19T16:21:30.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:30.077 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:21:30.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122054 2026-02-19T16:21:30.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122054 2026-02-19T16:21:30.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622 4-98784247832 5-335007449098 6-377957122054' 2026-02-19T16:21:30.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:30.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:21:30.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757271 2026-02-19T16:21:30.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757271 2026-02-19T16:21:30.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622 4-98784247832 5-335007449098 6-377957122054 7-163208757271' 2026-02-19T16:21:30.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:30.205 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:21:30.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593750 2026-02-19T16:21:30.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593750 2026-02-19T16:21:30.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622 4-98784247832 5-335007449098 6-377957122054 7-163208757271 8-184683593750' 2026-02-19T16:21:30.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:30.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:21:30.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958531 2026-02-19T16:21:30.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958531 2026-02-19T16:21:30.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622 4-98784247832 5-335007449098 6-377957122054 7-163208757271 8-184683593750 9-399431958531' 2026-02-19T16:21:30.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:30.336 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:21:30.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299413 2026-02-19T16:21:30.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299413 2026-02-19T16:21:30.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-313532612622 4-98784247832 5-335007449098 6-377957122054 7-163208757271 8-184683593750 9-399431958531 10-223338299413' 2026-02-19T16:21:30.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:30.399 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836506 2026-02-19T16:21:30.399 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:30.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:21:30.400 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836506 2026-02-19T16:21:30.401 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:30.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836506 2026-02-19T16:21:30.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836506' 2026-02-19T16:21:30.401 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 21474836506 2026-02-19T16:21:30.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:30.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836504 -lt 21474836506 2026-02-19T16:21:30.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:21:31.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:21:31.631 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:31.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836506 -lt 21474836506 2026-02-19T16:21:31.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:31.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672986 2026-02-19T16:21:31.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:31.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:21:31.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672986 2026-02-19T16:21:31.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:31.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672986 2026-02-19T16:21:31.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672986' 2026-02-19T16:21:31.860 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 42949672986 2026-02-19T16:21:31.860 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:21:32.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672986 -lt 42949672986 2026-02-19T16:21:32.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:32.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542169 2026-02-19T16:21:32.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:32.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:21:32.085 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542169 2026-02-19T16:21:32.085 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:32.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542169 2026-02-19T16:21:32.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542169' 2026-02-19T16:21:32.086 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 60129542169 2026-02-19T16:21:32.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:21:32.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542169 -lt 60129542169 2026-02-19T16:21:32.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:32.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-313532612622 2026-02-19T16:21:32.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:32.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:21:32.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612622 2026-02-19T16:21:32.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:32.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=313532612622 2026-02-19T16:21:32.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 313532612622' 2026-02-19T16:21:32.314 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 313532612622 2026-02-19T16:21:32.315 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:21:32.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 313532612622 -lt 313532612622 2026-02-19T16:21:32.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:32.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247832 2026-02-19T16:21:32.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:32.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:21:32.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247832 2026-02-19T16:21:32.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:32.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247832 2026-02-19T16:21:32.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247832' 2026-02-19T16:21:32.547 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.4 seq 98784247832 2026-02-19T16:21:32.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:21:32.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247832 -lt 98784247832 2026-02-19T16:21:32.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:32.777 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-335007449098 2026-02-19T16:21:32.777 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:32.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:21:32.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-335007449098 2026-02-19T16:21:32.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:32.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449098 2026-02-19T16:21:32.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 335007449098' 2026-02-19T16:21:32.779 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.5 seq 335007449098 2026-02-19T16:21:32.780 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:21:33.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449099 -lt 335007449098 2026-02-19T16:21:33.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:33.007 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-377957122054 2026-02-19T16:21:33.007 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:33.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:21:33.008 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-377957122054 2026-02-19T16:21:33.008 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:33.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=377957122054 2026-02-19T16:21:33.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 377957122054' 2026-02-19T16:21:33.009 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.6 seq 377957122054 2026-02-19T16:21:33.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:21:33.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 377957122055 -lt 377957122054 2026-02-19T16:21:33.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:33.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757271 2026-02-19T16:21:33.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:33.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:21:33.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757271 2026-02-19T16:21:33.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:33.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757271 2026-02-19T16:21:33.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757271' 2026-02-19T16:21:33.240 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.7 seq 163208757271 2026-02-19T16:21:33.240 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:21:33.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757271 -lt 163208757271 2026-02-19T16:21:33.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:33.467 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593750 2026-02-19T16:21:33.467 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:33.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:21:33.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593750 2026-02-19T16:21:33.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:33.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593750 2026-02-19T16:21:33.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593750' 2026-02-19T16:21:33.470 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.8 seq 184683593750 2026-02-19T16:21:33.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:21:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593751 -lt 184683593750 2026-02-19T16:21:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:33.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-399431958531 2026-02-19T16:21:33.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:33.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:21:33.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-399431958531 2026-02-19T16:21:33.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:33.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958531 2026-02-19T16:21:33.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 399431958531' 2026-02-19T16:21:33.698 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.9 seq 399431958531 2026-02-19T16:21:33.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:21:33.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958532 -lt 399431958531 2026-02-19T16:21:33.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:33.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299413 2026-02-19T16:21:33.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:33.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:21:33.934 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299413 2026-02-19T16:21:33.934 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:33.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299413 2026-02-19T16:21:33.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299413' 2026-02-19T16:21:33.935 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.10 seq 223338299413 2026-02-19T16:21:33.935 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:21:34.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299414 -lt 223338299413 2026-02-19T16:21:34.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:21:34.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:34.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:21:34.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:21:34.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T16:21:34.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:21:34.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:21:34.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:21:35.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T16:21:35.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:21:35.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:21:35.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:338: TEST_chunk_mapping: delete_pool remap-pool 2026-02-19T16:21:35.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-19T16:21:35.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-19T16:21:35.291 INFO:tasks.workunit.client.0.trial127.stderr:pool 'remap-pool' does not exist 2026-02-19T16:21:35.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:339: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-19T16:21:35.598 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile remap-profile does not exist 2026-02-19T16:21:35.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T16:21:35.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-19T16:21:35.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-19T16:21:35.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=isa 2026-02-19T16:21:35.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-19T16:21:35.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-19T16:21:35.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-19T16:21:35.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-19T16:21:35.607 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-19T16:21:35.608 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-19T16:21:35.608 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-19T16:21:35.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-19T16:21:35.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-19T16:21:35.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-19T16:21:35.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-19T16:21:36.179 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-19T16:21:36.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-19T16:21:36.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-19T16:21:36.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-19T16:21:36.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-19T16:21:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-19T16:21:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-19T16:21:36.852 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-isa' already exists 2026-02-19T16:21:36.861 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:21:37.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-19T16:21:37.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T16:21:37.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T16:21:37.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T16:21:38.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:21:38.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:38.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T16:21:38.082 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-19T16:21:38.083 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-19T16:21:38.083 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:21:38.083 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-19T16:21:38.083 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:38.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T16:21:38.314 INFO:tasks.workunit.client.0.trial127.stderr:10 2026-02-19T16:21:38.314 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:38.314 INFO:tasks.workunit.client.0.trial127.stderr:7' 2026-02-19T16:21:38.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-19T16:21:38.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-19T16:21:38.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T16:21:38.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-19T16:21:38.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-19T16:21:38.585 INFO:tasks.workunit.client.0.trial127.stderr:osd.7 is already out. 2026-02-19T16:21:38.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T16:21:43.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-19T16:21:43.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-19T16:21:43.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:21:43.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-19T16:21:43.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-19T16:21:43.597 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:43.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T16:21:43.827 INFO:tasks.workunit.client.0.trial127.stderr:10 2026-02-19T16:21:43.827 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:43.827 INFO:tasks.workunit.client.0.trial127.stderr:1' 2026-02-19T16:21:43.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-19T16:21:43.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:21:43.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-19T16:21:44.075 INFO:tasks.workunit.client.0.trial127.stderr:osd.7 is already in. 2026-02-19T16:21:44.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:21:44.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-19T16:21:44.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-19T16:21:44.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-19T16:21:44.376 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-isa' does not exist 2026-02-19T16:21:44.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T16:21:44.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-19T16:21:44.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-19T16:21:44.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-19T16:21:44.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T16:21:44.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T16:21:44.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:44.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T16:21:44.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T16:21:44.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:21:44.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:44.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T16:21:44.427 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-19T16:21:44.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T16:21:44.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:21:44.428 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T16:21:44.428 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:44.658 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-19T16:21:44.658 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:44.658 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:21:44.658 INFO:tasks.workunit.client.0.trial127.stderr:9' 2026-02-19T16:21:44.658 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-19T16:21:44.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-19T16:21:44.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T16:21:44.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-19T16:21:44.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-19T16:21:44.906 INFO:tasks.workunit.client.0.trial127.stderr:osd.9 is already out. 2026-02-19T16:21:44.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T16:21:49.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-19T16:21:49.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T16:21:49.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:21:49.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-19T16:21:49.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T16:21:49.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:50.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-19T16:21:50.158 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:21:50.158 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:21:50.158 INFO:tasks.workunit.client.0.trial127.stderr:5' 2026-02-19T16:21:50.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-19T16:21:50.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:21:50.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:50.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-19T16:21:50.451 INFO:tasks.workunit.client.0.trial127.stderr:osd.9 is already in. 2026-02-19T16:21:50.460 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:21:50.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-19T16:21:50.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-19T16:21:50.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-19T16:21:50.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-19T16:21:50.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-19T16:21:51.150 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:21:51.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:21:52.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-19T16:21:52.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T16:21:52.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-19T16:21:52.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T16:21:52.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:52.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T16:21:52.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:52.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T16:21:52.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:52.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T16:21:52.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:21:52.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T16:21:52.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T16:21:52.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:21:52.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:52.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T16:21:52.200 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-19T16:21:52.200 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:21:52.200 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:21:52.201 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T16:21:52.201 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:52.432 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T16:21:52.432 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:52.432 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:52.432 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:21:52.432 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:21:52.432 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:21:52.433 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-19T16:21:52.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-19T16:21:52.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T16:21:52.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-19T16:21:52.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-19T16:21:52.674 INFO:tasks.workunit.client.0.trial127.stderr:osd.10 is already out. 2026-02-19T16:21:52.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T16:21:57.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-19T16:21:57.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:21:57.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-19T16:21:57.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:21:57.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T16:21:57.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:21:57.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T16:21:57.918 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:57.919 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:57.919 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:21:57.919 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:21:57.919 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:21:57.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-19T16:21:57.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:21:57.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:57.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-19T16:21:58.165 INFO:tasks.workunit.client.0.trial127.stderr:osd.10 is already in. 2026-02-19T16:21:58.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T16:21:58.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-19T16:21:58.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T16:21:58.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-19T16:21:58.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T16:21:58.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:21:58.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:21:58.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-19T16:21:58.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-19T16:21:58.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:21:58.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:21:58.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:21:58.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:21:58.396 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:21:58.396 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:21:58.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:21:58.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:21:58.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:21:58.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:21:58.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:21:58.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:21:58.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:21:58.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:21:58.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:21:58.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:21:58.700 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:21:58.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:58.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:21:58.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-19T16:21:58.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-19T16:21:58.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514' 2026-02-19T16:21:58.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:58.764 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:21:58.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672994 2026-02-19T16:21:58.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672994 2026-02-19T16:21:58.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994' 2026-02-19T16:21:58.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:58.830 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:21:58.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542177 2026-02-19T16:21:58.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542177 2026-02-19T16:21:58.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177' 2026-02-19T16:21:58.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:58.896 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:21:58.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612630 2026-02-19T16:21:58.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612630 2026-02-19T16:21:58.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630' 2026-02-19T16:21:58.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:58.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:21:59.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247840 2026-02-19T16:21:59.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247840 2026-02-19T16:21:59.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630 4-98784247840' 2026-02-19T16:21:59.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:59.027 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:21:59.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449106 2026-02-19T16:21:59.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449106 2026-02-19T16:21:59.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630 4-98784247840 5-335007449106' 2026-02-19T16:21:59.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:59.091 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:21:59.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122062 2026-02-19T16:21:59.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122062 2026-02-19T16:21:59.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630 4-98784247840 5-335007449106 6-377957122062' 2026-02-19T16:21:59.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:59.154 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:21:59.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757278 2026-02-19T16:21:59.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757278 2026-02-19T16:21:59.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630 4-98784247840 5-335007449106 6-377957122062 7-163208757278' 2026-02-19T16:21:59.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:59.218 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:21:59.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593758 2026-02-19T16:21:59.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593758 2026-02-19T16:21:59.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630 4-98784247840 5-335007449106 6-377957122062 7-163208757278 8-184683593758' 2026-02-19T16:21:59.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:59.284 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:21:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958539 2026-02-19T16:21:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958539 2026-02-19T16:21:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630 4-98784247840 5-335007449106 6-377957122062 7-163208757278 8-184683593758 9-399431958539' 2026-02-19T16:21:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:21:59.348 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:21:59.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299421 2026-02-19T16:21:59.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299421 2026-02-19T16:21:59.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672994 2-60129542177 3-313532612630 4-98784247840 5-335007449106 6-377957122062 7-163208757278 8-184683593758 9-399431958539 10-223338299421' 2026-02-19T16:21:59.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:59.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836514 2026-02-19T16:21:59.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:59.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:21:59.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-19T16:21:59.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:59.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836514 2026-02-19T16:21:59.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-19T16:21:59.416 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 21474836514 2026-02-19T16:21:59.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:21:59.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836514 -lt 21474836514 2026-02-19T16:21:59.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:59.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672994 2026-02-19T16:21:59.644 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:59.644 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:21:59.645 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672994 2026-02-19T16:21:59.645 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:59.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672994 2026-02-19T16:21:59.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672994' 2026-02-19T16:21:59.647 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 42949672994 2026-02-19T16:21:59.647 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:21:59.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672994 -lt 42949672994 2026-02-19T16:21:59.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:21:59.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542177 2026-02-19T16:21:59.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:21:59.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:21:59.874 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542177 2026-02-19T16:21:59.874 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:21:59.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542177 2026-02-19T16:21:59.875 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 60129542177 2026-02-19T16:21:59.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542177' 2026-02-19T16:21:59.876 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:22:00.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542177 -lt 60129542177 2026-02-19T16:22:00.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:00.101 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-313532612630 2026-02-19T16:22:00.101 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:00.102 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:22:00.103 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612630 2026-02-19T16:22:00.103 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:00.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=313532612630 2026-02-19T16:22:00.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 313532612630' 2026-02-19T16:22:00.104 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 313532612630 2026-02-19T16:22:00.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:22:00.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 313532612630 -lt 313532612630 2026-02-19T16:22:00.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:00.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247840 2026-02-19T16:22:00.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:00.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:22:00.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247840 2026-02-19T16:22:00.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:00.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247840 2026-02-19T16:22:00.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247840' 2026-02-19T16:22:00.335 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 98784247840 2026-02-19T16:22:00.335 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:22:00.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247840 -lt 98784247840 2026-02-19T16:22:00.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:00.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-335007449106 2026-02-19T16:22:00.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:00.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:22:00.564 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-335007449106 2026-02-19T16:22:00.564 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:00.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449106 2026-02-19T16:22:00.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 335007449106' 2026-02-19T16:22:00.566 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 335007449106 2026-02-19T16:22:00.566 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:22:00.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449106 -lt 335007449106 2026-02-19T16:22:00.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:00.794 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-377957122062 2026-02-19T16:22:00.794 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:00.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:22:00.796 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-377957122062 2026-02-19T16:22:00.796 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:00.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=377957122062 2026-02-19T16:22:00.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 377957122062' 2026-02-19T16:22:00.797 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 377957122062 2026-02-19T16:22:00.798 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:22:01.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 377957122062 -lt 377957122062 2026-02-19T16:22:01.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:01.023 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757278 2026-02-19T16:22:01.023 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:01.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:22:01.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757278 2026-02-19T16:22:01.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:01.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757278 2026-02-19T16:22:01.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757278' 2026-02-19T16:22:01.026 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.7 seq 163208757278 2026-02-19T16:22:01.026 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:22:01.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757279 -lt 163208757278 2026-02-19T16:22:01.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:01.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593758 2026-02-19T16:22:01.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:01.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:22:01.252 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593758 2026-02-19T16:22:01.252 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:01.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593758 2026-02-19T16:22:01.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593758' 2026-02-19T16:22:01.254 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.8 seq 184683593758 2026-02-19T16:22:01.254 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:22:01.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593758 -lt 184683593758 2026-02-19T16:22:01.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:01.481 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-399431958539 2026-02-19T16:22:01.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:01.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:22:01.483 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-399431958539 2026-02-19T16:22:01.483 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:01.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958539 2026-02-19T16:22:01.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 399431958539' 2026-02-19T16:22:01.485 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.9 seq 399431958539 2026-02-19T16:22:01.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:22:01.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958539 -lt 399431958539 2026-02-19T16:22:01.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:01.715 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299421 2026-02-19T16:22:01.715 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:01.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:22:01.717 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299421 2026-02-19T16:22:01.717 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:01.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299421 2026-02-19T16:22:01.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299421' 2026-02-19T16:22:01.719 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.10 seq 223338299421 2026-02-19T16:22:01.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:22:01.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299421 -lt 223338299421 2026-02-19T16:22:01.945 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:22:01.945 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:22:01.946 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:22:02.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T16:22:02.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:22:02.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:22:02.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:22:02.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:22:02.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:22:02.271 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:22:02.271 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:22:02.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T16:22:02.500 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:22:02.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:22:02.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:22:02.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T16:22:02.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:22:02.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:22:02.816 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T16:22:02.816 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:22:02.816 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:02.817 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T16:22:02.817 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:03.047 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T16:22:03.047 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-19T16:22:03.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-19T16:22:03.297 INFO:tasks.workunit.client.0.trial127.stderr:osd.5 is already out. 2026-02-19T16:22:03.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-19T16:22:03.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-19T16:22:03.605 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 is already out. 2026-02-19T16:22:03.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-19T16:22:03.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:22:03.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:22:03.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:22:03.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:22:03.615 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:22:03.615 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:22:03.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:22:03.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:22:03.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:22:03.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:22:03.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:22:03.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:22:03.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:22:03.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:22:03.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:22:03.922 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:22:03.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:22:03.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:03.924 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:22:03.990 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836517 2026-02-19T16:22:03.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836517 2026-02-19T16:22:03.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517' 2026-02-19T16:22:03.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:03.991 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:22:04.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672997 2026-02-19T16:22:04.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672997 2026-02-19T16:22:04.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997' 2026-02-19T16:22:04.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.055 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:22:04.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542180 2026-02-19T16:22:04.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542180 2026-02-19T16:22:04.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180' 2026-02-19T16:22:04.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:22:04.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612633 2026-02-19T16:22:04.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612633 2026-02-19T16:22:04.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633' 2026-02-19T16:22:04.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.184 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:22:04.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247843 2026-02-19T16:22:04.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247843 2026-02-19T16:22:04.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633 4-98784247843' 2026-02-19T16:22:04.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.248 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:22:04.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449109 2026-02-19T16:22:04.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449109 2026-02-19T16:22:04.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633 4-98784247843 5-335007449109' 2026-02-19T16:22:04.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:22:04.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122065 2026-02-19T16:22:04.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122065 2026-02-19T16:22:04.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633 4-98784247843 5-335007449109 6-377957122065' 2026-02-19T16:22:04.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.376 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:22:04.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757282 2026-02-19T16:22:04.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757282 2026-02-19T16:22:04.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633 4-98784247843 5-335007449109 6-377957122065 7-163208757282' 2026-02-19T16:22:04.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.441 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:22:04.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593761 2026-02-19T16:22:04.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593761 2026-02-19T16:22:04.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633 4-98784247843 5-335007449109 6-377957122065 7-163208757282 8-184683593761' 2026-02-19T16:22:04.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:22:04.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958542 2026-02-19T16:22:04.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958542 2026-02-19T16:22:04.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633 4-98784247843 5-335007449109 6-377957122065 7-163208757282 8-184683593761 9-399431958542' 2026-02-19T16:22:04.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:04.572 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:22:04.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299424 2026-02-19T16:22:04.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299424 2026-02-19T16:22:04.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-313532612633 4-98784247843 5-335007449109 6-377957122065 7-163208757282 8-184683593761 9-399431958542 10-223338299424' 2026-02-19T16:22:04.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:04.637 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836517 2026-02-19T16:22:04.637 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:04.638 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:22:04.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836517 2026-02-19T16:22:04.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:04.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836517 2026-02-19T16:22:04.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-19T16:22:04.640 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 21474836517 2026-02-19T16:22:04.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:22:04.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836515 -lt 21474836517 2026-02-19T16:22:04.866 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:22:05.866 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:22:05.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:22:06.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836517 -lt 21474836517 2026-02-19T16:22:06.101 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:06.101 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672997 2026-02-19T16:22:06.101 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:06.102 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:22:06.102 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672997 2026-02-19T16:22:06.102 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:06.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672997 2026-02-19T16:22:06.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672997' 2026-02-19T16:22:06.104 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 42949672997 2026-02-19T16:22:06.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:22:06.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672997 -lt 42949672997 2026-02-19T16:22:06.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:06.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542180 2026-02-19T16:22:06.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:06.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:22:06.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542180 2026-02-19T16:22:06.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:06.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542180 2026-02-19T16:22:06.332 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 60129542180 2026-02-19T16:22:06.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542180' 2026-02-19T16:22:06.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:22:06.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542180 -lt 60129542180 2026-02-19T16:22:06.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:06.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-313532612633 2026-02-19T16:22:06.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:06.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:22:06.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612633 2026-02-19T16:22:06.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:06.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=313532612633 2026-02-19T16:22:06.557 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 313532612633 2026-02-19T16:22:06.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 313532612633' 2026-02-19T16:22:06.558 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:22:06.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 313532612633 -lt 313532612633 2026-02-19T16:22:06.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:06.782 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247843 2026-02-19T16:22:06.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:06.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:22:06.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247843 2026-02-19T16:22:06.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:06.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247843 2026-02-19T16:22:06.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247843' 2026-02-19T16:22:06.785 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 98784247843 2026-02-19T16:22:06.786 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:22:07.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247843 -lt 98784247843 2026-02-19T16:22:07.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:07.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-335007449109 2026-02-19T16:22:07.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:07.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:22:07.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-335007449109 2026-02-19T16:22:07.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:07.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449109 2026-02-19T16:22:07.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 335007449109' 2026-02-19T16:22:07.019 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 335007449109 2026-02-19T16:22:07.019 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:22:07.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449109 -lt 335007449109 2026-02-19T16:22:07.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:07.243 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-377957122065 2026-02-19T16:22:07.243 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:07.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:22:07.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-377957122065 2026-02-19T16:22:07.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:07.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=377957122065 2026-02-19T16:22:07.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 377957122065' 2026-02-19T16:22:07.247 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 377957122065 2026-02-19T16:22:07.247 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:22:07.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 377957122066 -lt 377957122065 2026-02-19T16:22:07.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:07.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757282 2026-02-19T16:22:07.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:07.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:22:07.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757282 2026-02-19T16:22:07.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:07.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757282 2026-02-19T16:22:07.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757282' 2026-02-19T16:22:07.478 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.7 seq 163208757282 2026-02-19T16:22:07.479 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:22:07.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757282 -lt 163208757282 2026-02-19T16:22:07.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:07.706 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593761 2026-02-19T16:22:07.706 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:07.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:22:07.707 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593761 2026-02-19T16:22:07.707 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:07.708 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593761 2026-02-19T16:22:07.708 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593761' 2026-02-19T16:22:07.709 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.8 seq 184683593761 2026-02-19T16:22:07.709 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:22:07.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593762 -lt 184683593761 2026-02-19T16:22:07.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:07.955 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-399431958542 2026-02-19T16:22:07.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:07.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:22:07.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-399431958542 2026-02-19T16:22:07.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:07.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958542 2026-02-19T16:22:07.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 399431958542' 2026-02-19T16:22:07.958 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.9 seq 399431958542 2026-02-19T16:22:07.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:22:08.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958542 -lt 399431958542 2026-02-19T16:22:08.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:08.185 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299424 2026-02-19T16:22:08.186 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:08.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:22:08.187 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299424 2026-02-19T16:22:08.187 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:08.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299424 2026-02-19T16:22:08.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299424' 2026-02-19T16:22:08.189 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.10 seq 223338299424 2026-02-19T16:22:08.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:22:08.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299425 -lt 223338299424 2026-02-19T16:22:08.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:22:08.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:22:08.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:22:08.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T16:22:08.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:22:08.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:22:08.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:22:08.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:22:08.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:22:08.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:22:08.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:22:08.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T16:22:08.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:22:08.960 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:22:08.960 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:22:09.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T16:22:09.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:09.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:09.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-19T16:22:09.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T16:22:09.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-19T16:22:09.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:22:09.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:22:09.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-19T16:22:09.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-19T16:22:09.984 INFO:tasks.workunit.client.0.trial127.stderr:osd.5 is already in. 2026-02-19T16:22:09.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-19T16:22:09.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-19T16:22:10.288 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 is already in. 2026-02-19T16:22:10.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-19T16:22:10.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:22:10.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:22:10.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:22:10.297 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:22:10.298 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:22:10.298 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:22:10.298 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:22:10.298 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:22:10.298 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:22:10.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:22:10.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:22:10.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:22:10.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:22:10.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:22:10.382 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:22:10.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:22:10.607 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:22:10.607 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:22:10.607 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:10.607 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:22:10.607 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:22:10.607 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:22:10.608 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:22:10.608 INFO:tasks.workunit.client.0.trial127.stderr:8 2026-02-19T16:22:10.608 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:10.608 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:22:10.608 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:22:10.608 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:10.608 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:22:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836520 2026-02-19T16:22:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836520 2026-02-19T16:22:10.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520' 2026-02-19T16:22:10.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:10.672 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:22:10.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949673000 2026-02-19T16:22:10.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949673000 2026-02-19T16:22:10.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000' 2026-02-19T16:22:10.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:10.737 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:22:10.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542183 2026-02-19T16:22:10.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542183 2026-02-19T16:22:10.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183' 2026-02-19T16:22:10.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:10.799 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:22:10.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612636 2026-02-19T16:22:10.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612636 2026-02-19T16:22:10.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636' 2026-02-19T16:22:10.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:10.865 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:22:10.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247846 2026-02-19T16:22:10.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247846 2026-02-19T16:22:10.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636 4-98784247846' 2026-02-19T16:22:10.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:10.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:22:10.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449113 2026-02-19T16:22:10.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449113 2026-02-19T16:22:10.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636 4-98784247846 5-335007449113' 2026-02-19T16:22:10.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:10.991 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:22:11.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122069 2026-02-19T16:22:11.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122069 2026-02-19T16:22:11.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636 4-98784247846 5-335007449113 6-377957122069' 2026-02-19T16:22:11.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:11.056 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:22:11.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757285 2026-02-19T16:22:11.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757285 2026-02-19T16:22:11.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636 4-98784247846 5-335007449113 6-377957122069 7-163208757285' 2026-02-19T16:22:11.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:11.122 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:22:11.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593764 2026-02-19T16:22:11.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593764 2026-02-19T16:22:11.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636 4-98784247846 5-335007449113 6-377957122069 7-163208757285 8-184683593764' 2026-02-19T16:22:11.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:11.186 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-19T16:22:11.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958545 2026-02-19T16:22:11.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958545 2026-02-19T16:22:11.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636 4-98784247846 5-335007449113 6-377957122069 7-163208757285 8-184683593764 9-399431958545' 2026-02-19T16:22:11.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:11.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-19T16:22:11.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299427 2026-02-19T16:22:11.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299427 2026-02-19T16:22:11.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-313532612636 4-98784247846 5-335007449113 6-377957122069 7-163208757285 8-184683593764 9-399431958545 10-223338299427' 2026-02-19T16:22:11.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:11.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836520 2026-02-19T16:22:11.315 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:11.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:22:11.316 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836520 2026-02-19T16:22:11.316 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:11.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836520 2026-02-19T16:22:11.317 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 21474836520 2026-02-19T16:22:11.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836520' 2026-02-19T16:22:11.318 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:22:11.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836520 -lt 21474836520 2026-02-19T16:22:11.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:11.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949673000 2026-02-19T16:22:11.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:11.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:22:11.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949673000 2026-02-19T16:22:11.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:11.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949673000 2026-02-19T16:22:11.548 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 42949673000 2026-02-19T16:22:11.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949673000' 2026-02-19T16:22:11.548 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:22:11.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949673000 -lt 42949673000 2026-02-19T16:22:11.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:11.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542183 2026-02-19T16:22:11.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:11.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:22:11.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542183 2026-02-19T16:22:11.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:11.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542183 2026-02-19T16:22:11.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542183' 2026-02-19T16:22:11.779 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 60129542183 2026-02-19T16:22:11.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:22:12.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542183 -lt 60129542183 2026-02-19T16:22:12.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:12.015 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-313532612636 2026-02-19T16:22:12.015 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:12.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:22:12.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612636 2026-02-19T16:22:12.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:12.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=313532612636 2026-02-19T16:22:12.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 313532612636' 2026-02-19T16:22:12.018 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 313532612636 2026-02-19T16:22:12.018 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:22:12.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 313532612636 -lt 313532612636 2026-02-19T16:22:12.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:12.248 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247846 2026-02-19T16:22:12.248 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:12.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:22:12.250 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247846 2026-02-19T16:22:12.250 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:12.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247846 2026-02-19T16:22:12.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247846' 2026-02-19T16:22:12.251 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 98784247846 2026-02-19T16:22:12.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:22:12.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247846 -lt 98784247846 2026-02-19T16:22:12.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:12.478 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-335007449113 2026-02-19T16:22:12.479 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:12.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:22:12.480 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-335007449113 2026-02-19T16:22:12.480 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:12.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449113 2026-02-19T16:22:12.481 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 335007449113 2026-02-19T16:22:12.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 335007449113' 2026-02-19T16:22:12.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:22:12.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449113 -lt 335007449113 2026-02-19T16:22:12.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:12.711 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-377957122069 2026-02-19T16:22:12.712 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:12.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:22:12.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-377957122069 2026-02-19T16:22:12.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:12.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=377957122069 2026-02-19T16:22:12.714 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 377957122069 2026-02-19T16:22:12.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 377957122069' 2026-02-19T16:22:12.715 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:22:12.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 377957122069 -lt 377957122069 2026-02-19T16:22:12.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:12.942 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757285 2026-02-19T16:22:12.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:12.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:22:12.944 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757285 2026-02-19T16:22:12.944 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:12.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757285 2026-02-19T16:22:12.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757285' 2026-02-19T16:22:12.945 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.7 seq 163208757285 2026-02-19T16:22:12.946 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:22:13.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757285 -lt 163208757285 2026-02-19T16:22:13.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:13.174 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593764 2026-02-19T16:22:13.174 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:13.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:22:13.175 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593764 2026-02-19T16:22:13.175 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:13.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593764 2026-02-19T16:22:13.177 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.8 seq 184683593764 2026-02-19T16:22:13.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593764' 2026-02-19T16:22:13.177 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:22:13.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593764 -lt 184683593764 2026-02-19T16:22:13.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:13.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-399431958545 2026-02-19T16:22:13.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:13.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-19T16:22:13.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-399431958545 2026-02-19T16:22:13.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:13.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958545 2026-02-19T16:22:13.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 399431958545' 2026-02-19T16:22:13.405 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.9 seq 399431958545 2026-02-19T16:22:13.405 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T16:22:13.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958546 -lt 399431958545 2026-02-19T16:22:13.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:13.636 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299427 2026-02-19T16:22:13.636 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:13.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-19T16:22:13.638 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299427 2026-02-19T16:22:13.638 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:13.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299427 2026-02-19T16:22:13.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299427' 2026-02-19T16:22:13.640 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.10 seq 223338299427 2026-02-19T16:22:13.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T16:22:13.868 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299428 -lt 223338299427 2026-02-19T16:22:13.869 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:22:13.870 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:22:13.870 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:22:14.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T16:22:14.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:22:14.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:22:14.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:22:14.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:22:14.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:22:14.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:22:14.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:22:14.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T16:22:14.412 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:22:14.413 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:22:14.413 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:22:14.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T16:22:14.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:22:14.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:22:14.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T16:22:14.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:22:14.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:14.728 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T16:22:14.728 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr:10' 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-19T16:22:14.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:22:14.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-19T16:22:14.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-19T16:22:14.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:22:15.203 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:22:15.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-19T16:22:15.489 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-19T16:22:15.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T16:22:15.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-19T16:22:15.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-19T16:22:15.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-19T16:22:15.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-19T16:22:15.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-19T16:22:15.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-19T16:22:15.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-19T16:22:16.159 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-lrc-a' already exists 2026-02-19T16:22:16.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:22:17.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:17.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T16:22:17.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:17.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T16:22:17.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T16:22:17.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:22:17.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:22:17.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T16:22:17.204 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-19T16:22:17.204 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-19T16:22:17.205 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:17.205 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-19T16:22:17.205 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:17.438 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T16:22:17.438 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:17.439 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:22:17.439 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-19T16:22:17.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-19T16:22:17.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T16:22:17.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-19T16:22:17.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-19T16:22:17.690 INFO:tasks.workunit.client.0.trial127.stderr:osd.3 is already out. 2026-02-19T16:22:17.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T16:22:22.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-19T16:22:22.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-19T16:22:22.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:22.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-19T16:22:22.700 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-19T16:22:22.700 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:22.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T16:22:22.930 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:22.930 INFO:tasks.workunit.client.0.trial127.stderr:7' 2026-02-19T16:22:22.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-19T16:22:22.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:22:22.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:22:22.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-19T16:22:23.179 INFO:tasks.workunit.client.0.trial127.stderr:osd.3 is already in. 2026-02-19T16:22:23.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:22:23.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-19T16:22:23.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-19T16:22:23.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-19T16:22:23.481 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-lrc-a' does not exist 2026-02-19T16:22:23.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-19T16:22:23.769 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-19T16:22:23.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T16:22:23.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-19T16:22:23.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-19T16:22:23.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-19T16:22:23.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-19T16:22:23.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-19T16:22:24.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-19T16:22:24.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-19T16:22:24.445 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-lrc' already exists 2026-02-19T16:22:24.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:22:25.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-19T16:22:25.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T16:22:25.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T16:22:25.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T16:22:25.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:22:25.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:22:25.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T16:22:25.494 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-19T16:22:25.494 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-19T16:22:25.494 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:25.495 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-19T16:22:25.495 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:10 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:7' 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-19T16:22:25.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-19T16:22:25.976 INFO:tasks.workunit.client.0.trial127.stderr:osd.7 is already out. 2026-02-19T16:22:25.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T16:22:30.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-19T16:22:30.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-19T16:22:30.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:30.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-19T16:22:30.989 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-19T16:22:30.989 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:31.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-19T16:22:31.218 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:22:31.218 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:22:31.218 INFO:tasks.workunit.client.0.trial127.stderr:10 2026-02-19T16:22:31.218 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:22:31.218 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:31.218 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:31.219 INFO:tasks.workunit.client.0.trial127.stderr:4' 2026-02-19T16:22:31.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-19T16:22:31.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:22:31.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:22:31.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-19T16:22:31.569 INFO:tasks.workunit.client.0.trial127.stderr:osd.7 is already in. 2026-02-19T16:22:31.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:22:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-19T16:22:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-19T16:22:31.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-19T16:22:31.868 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-lrc' does not exist 2026-02-19T16:22:31.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-19T16:22:32.154 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile profile-lrc does not exist 2026-02-19T16:22:32.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T16:22:32.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-19T16:22:32.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-19T16:22:32.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-19T16:22:32.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-19T16:22:32.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-19T16:22:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-19T16:22:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-19T16:22:32.828 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-shec' already exists 2026-02-19T16:22:32.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:22:33.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T16:22:33.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T16:22:33.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T16:22:33.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T16:22:33.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:22:33.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:22:33.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T16:22:33.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-19T16:22:33.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-19T16:22:33.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:33.877 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-19T16:22:33.877 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:34.106 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-19T16:22:34.107 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:34.107 INFO:tasks.workunit.client.0.trial127.stderr:1' 2026-02-19T16:22:34.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-19T16:22:34.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-19T16:22:34.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T16:22:34.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-19T16:22:34.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-19T16:22:34.357 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 is already out. 2026-02-19T16:22:34.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T16:22:39.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-19T16:22:39.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-19T16:22:39.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T16:22:39.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-19T16:22:39.368 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-19T16:22:39.368 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:22:39.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-19T16:22:39.603 INFO:tasks.workunit.client.0.trial127.stderr:9 2026-02-19T16:22:39.603 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:22:39.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-19T16:22:39.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-19T16:22:39.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T16:22:39.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-19T16:22:39.903 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 is already in. 2026-02-19T16:22:39.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T16:22:39.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-19T16:22:39.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-19T16:22:39.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-19T16:22:40.204 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-shec' does not exist 2026-02-19T16:22:40.213 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-19T16:22:40.493 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile profile-shec does not exist 2026-02-19T16:22:40.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-19T16:22:40.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-19T16:22:40.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-19T16:22:40.777 INFO:tasks.workunit.client.0.trial127.stderr:pool 'ecpool' does not exist 2026-02-19T16:22:40.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-19T16:22:40.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-19T16:22:40.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:22:40.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-19T16:22:40.787 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:22:40.787 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:22:40.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:22:40.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:22:40.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:22:40.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:22:40.898 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:22:40.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:22:40.899 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:22:40.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:22:40.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:22:40.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:22:40.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:22:40.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:22:40.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:22:40.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:22:40.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:22:40.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:22:40.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-19T16:22:40.962 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:22:40.963 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:40.963 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:22:40.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.67892 2026-02-19T16:22:40.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:22:40.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code 0 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:22:40.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:22:40.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:22:40.966 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:22:40.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:22:40.967 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:22:40.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:22:40.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:22:40.968 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:22:40.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:22:40.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:22:40.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:22:40.970 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:22:40.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:22:40.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T16:22:40.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-19T16:22:40.972 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:22:40.973 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:40.973 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.67892 2026-02-19T16:22:40.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.67892 2026-02-19T16:22:40.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:22:40.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:22:40.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-19T16:22:40.975 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T16:22:40.975 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T16:22:41.040 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-19T16:22:41.040 DEBUG:teuthology.orchestra.run.trial127:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=802bf111cd55e57f5842153d7ba2b8b964aa818a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-19T16:22:41.102 INFO:tasks.workunit.client.0.trial127.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-19T16:22:41.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-eio 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none ' 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-19T16:22:41.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:TEST_ec_object_attr_read_error 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:TEST_ec_recovery_multiple_errors 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:TEST_ec_recovery_multiple_objects 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:TEST_ec_recovery_unfound 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:TEST_ec_single_recovery_error 2026-02-19T16:22:41.109 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_get_bad_size_shard_0 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_get_bad_size_shard_1 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_get_subread_missing 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:22:41.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:22:41.111 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:22:41.111 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:22:41.112 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:22:41.112 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:22:41.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:22:41.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:22:41.113 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:22:41.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:22:41.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:22:41.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:22:41.114 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:22:41.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:22:41.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:22:41.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:22:41.117 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:22:41.117 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:41.117 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:41.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:22:41.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:22:41.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:22:41.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:22:41.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:22:41.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:41.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:41.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:22:41.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:22:41.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:22:41.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:22:41.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:22:41.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:22:41.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:22:41.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:41.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:41.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:41.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:22:41.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:22:41.167 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:41.168 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:41.168 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:22:41.168 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:22:41.168 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:22:41.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:22:41.210 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:22:41.210 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:22:41.210 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:22:41.210 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:22:41.210 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:22:41.210 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:22:41.210 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:22:41.211 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:22:41.211 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:41.211 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:41.211 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:22:41.211 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:22:41.212 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:22:41.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:22:41.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:22:41.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:22:41.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:22:41.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:22:41.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:22:41.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:22:41.351 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:22:41.351 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:22:41.351 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:22:41.351 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:22:41.351 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:41.351 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:41.351 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:41.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:22:41.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:22:41.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:22:41.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:22:41.463 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:22:41.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:22:42.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:22:42.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:22:42.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:22:42.472 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:22:42.472 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:42.472 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:42.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:22:42.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:22:42.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:22:42.512 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:22:42.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:22:42.517 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:22:41.156+0000 7ff5ad487c00 0 load: jerasure load: lrc 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:22:42.518 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-19T16:22:42.519 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-19T16:22:42.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:42.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:22:42.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:22:42.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=493a0d1d-2663-48a4-abeb-773bc77faef5 2026-02-19T16:22:42.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 493a0d1d-2663-48a4-abeb-773bc77faef5' 2026-02-19T16:22:42.523 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 493a0d1d-2663-48a4-abeb-773bc77faef5 2026-02-19T16:22:42.523 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:22:42.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDSOJdp5RLFHxAA69JUh4/5ahfRCCTShDjpSQ== 2026-02-19T16:22:42.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDSOJdp5RLFHxAA69JUh4/5ahfRCCTShDjpSQ=="}' 2026-02-19T16:22:42.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 493a0d1d-2663-48a4-abeb-773bc77faef5 -i td/test-erasure-eio/0/new.json 2026-02-19T16:22:42.641 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:22:42.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:22:42.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDSOJdp5RLFHxAA69JUh4/5ahfRCCTShDjpSQ== --osd-uuid 493a0d1d-2663-48a4-abeb-773bc77faef5 2026-02-19T16:22:42.667 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:42.665+0000 7f1ef4842780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:42.669 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:42.667+0000 7f1ef4842780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:42.670 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:42.668+0000 7f1ef4842780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:42.670 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:42.668+0000 7f1ef4842780 -1 bdev(0x3546c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:22:42.670 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:42.668+0000 7f1ef4842780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:22:43.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:22:43.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:22:43.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:22:43.088 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:22:43.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:22:43.193 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:22:43.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:22:43.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:22:43.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:22:43.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:22:43.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:22:43.211 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:43.209+0000 7fd27aa42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:43.212 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:43.211+0000 7fd27aa42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:43.213 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:43.212+0000 7fd27aa42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:43.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:22:43.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:22:43.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:22:43.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:22:43.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:22:43.295 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:22:43.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:43.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:22:43.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:43.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:22:43.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:22:43.431 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:43.428+0000 7fd27aa42780 -1 Falling back to public interface 2026-02-19T16:22:43.692 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:43.690+0000 7fd27aa42780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:22:44.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:22:44.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:44.393 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:22:44.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:22:44.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:44.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:22:44.622 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2386157277,v1:127.0.0.1:6803/2386157277] [v2:127.0.0.1:6804/2386157277,v1:127.0.0.1:6805/2386157277] exists,up 493a0d1d-2663-48a4-abeb-773bc77faef5 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:22:44.624 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:22:44.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9258334a-6402-4c47-9a33-4e504f33e615 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 9258334a-6402-4c47-9a33-4e504f33e615' 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 9258334a-6402-4c47-9a33-4e504f33e615 2026-02-19T16:22:44.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:22:44.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDUOJdpwc/oJRAAslPBF6sZjVI3IVOSq2y4ZQ== 2026-02-19T16:22:44.638 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDUOJdpwc/oJRAAslPBF6sZjVI3IVOSq2y4ZQ=="}' 2026-02-19T16:22:44.638 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9258334a-6402-4c47-9a33-4e504f33e615 -i td/test-erasure-eio/1/new.json 2026-02-19T16:22:44.866 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:22:44.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:22:44.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDUOJdpwc/oJRAAslPBF6sZjVI3IVOSq2y4ZQ== --osd-uuid 9258334a-6402-4c47-9a33-4e504f33e615 2026-02-19T16:22:44.893 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:44.891+0000 7f2617253780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:44.894 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:44.893+0000 7f2617253780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:44.895 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:44.894+0000 7f2617253780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:44.896 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:44.894+0000 7f2617253780 -1 bdev(0x2744c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:22:44.896 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:44.894+0000 7f2617253780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:22:45.230 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:22:45.230 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:22:45.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:22:45.231 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:22:45.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:22:45.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:22:45.570 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:22:45.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:22:45.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:22:45.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:22:45.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:22:45.587 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:45.585+0000 7ff8d7433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:45.588 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:45.587+0000 7ff8d7433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:45.589 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:45.587+0000 7ff8d7433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:45.753 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:45.752+0000 7ff8d7433780 -1 Falling back to public interface 2026-02-19T16:22:45.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:22:45.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:22:45.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:22:45.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:22:45.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:22:45.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:45.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:22:45.805 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:22:45.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:45.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:22:46.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:22:46.043 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:46.042+0000 7ff8d7433780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:22:47.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:22:47.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:47.034 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:22:47.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:22:47.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:47.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:22:47.265 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2676214838,v1:127.0.0.1:6811/2676214838] [v2:127.0.0.1:6812/2676214838,v1:127.0.0.1:6813/2676214838] exists,up 9258334a-6402-4c47-9a33-4e504f33e615 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:22:47.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:22:47.271 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:22:47.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d07406bc-20a1-447d-bd85-5f8053335109 2026-02-19T16:22:47.273 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 d07406bc-20a1-447d-bd85-5f8053335109 2026-02-19T16:22:47.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d07406bc-20a1-447d-bd85-5f8053335109' 2026-02-19T16:22:47.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:22:47.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDXOJdpU7qsEBAAHVOXm7PxhNtjIi7M9qkWjw== 2026-02-19T16:22:47.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDXOJdpU7qsEBAAHVOXm7PxhNtjIi7M9qkWjw=="}' 2026-02-19T16:22:47.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d07406bc-20a1-447d-bd85-5f8053335109 -i td/test-erasure-eio/2/new.json 2026-02-19T16:22:47.510 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:22:47.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:22:47.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDXOJdpU7qsEBAAHVOXm7PxhNtjIi7M9qkWjw== --osd-uuid d07406bc-20a1-447d-bd85-5f8053335109 2026-02-19T16:22:47.537 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:47.535+0000 7ff313e46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:47.538 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:47.537+0000 7ff313e46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:47.539 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:47.538+0000 7ff313e46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:47.539 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:47.538+0000 7ff313e46780 -1 bdev(0x2afac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:22:47.540 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:47.538+0000 7ff313e46780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:22:47.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:22:47.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:22:47.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:22:47.880 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:22:47.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:22:48.202 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:22:48.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:22:48.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:22:48.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:22:48.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:22:48.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:22:48.219 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:48.217+0000 7faa29630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:48.220 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:48.218+0000 7faa29630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:48.221 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:48.219+0000 7faa29630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:48.340 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:48.338+0000 7faa29630780 -1 Falling back to public interface 2026-02-19T16:22:48.435 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:22:48.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:22:48.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:22:48.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:22:48.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:22:48.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:48.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:22:48.436 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:22:48.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:48.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:22:48.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:22:48.712 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:48.710+0000 7faa29630780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:22:49.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:22:49.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:49.663 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:22:49.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:22:49.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:49.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:22:49.896 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3964611012,v1:127.0.0.1:6819/3964611012] [v2:127.0.0.1:6820/3964611012,v1:127.0.0.1:6821/3964611012] exists,up d07406bc-20a1-447d-bd85-5f8053335109 2026-02-19T16:22:49.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:22:49.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:22:49.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:22:49.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:22:49.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:22:49.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:22:49.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:22:49.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:22:49.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=24692fe7-23dc-4673-87cf-40c9c8b176c7 2026-02-19T16:22:49.904 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 24692fe7-23dc-4673-87cf-40c9c8b176c7 2026-02-19T16:22:49.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 24692fe7-23dc-4673-87cf-40c9c8b176c7' 2026-02-19T16:22:49.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:22:49.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDZOJdp9P84NhAAOeeJCXVYIDg+x7XgX0C3zg== 2026-02-19T16:22:49.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDZOJdp9P84NhAAOeeJCXVYIDg+x7XgX0C3zg=="}' 2026-02-19T16:22:49.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 24692fe7-23dc-4673-87cf-40c9c8b176c7 -i td/test-erasure-eio/3/new.json 2026-02-19T16:22:50.146 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:22:50.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:22:50.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDZOJdp9P84NhAAOeeJCXVYIDg+x7XgX0C3zg== --osd-uuid 24692fe7-23dc-4673-87cf-40c9c8b176c7 2026-02-19T16:22:50.174 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.173+0000 7f150824f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:50.176 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.174+0000 7f150824f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:50.177 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.175+0000 7f150824f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:50.177 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.175+0000 7f150824f780 -1 bdev(0x2a2ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:22:50.177 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.175+0000 7f150824f780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:22:50.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:22:50.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:22:50.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:22:50.514 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:22:50.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:22:50.834 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:22:50.834 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:22:50.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:22:50.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:22:50.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:22:50.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:22:50.851 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.849+0000 7f946582e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:50.852 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.850+0000 7f946582e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:50.853 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:50.851+0000 7f946582e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:51.004 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:51.002+0000 7f946582e780 -1 Falling back to public interface 2026-02-19T16:22:51.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:22:51.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:22:51.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:22:51.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:22:51.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:22:51.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:51.064 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:22:51.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:22:51.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:51.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:22:51.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:22:51.323 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:51.321+0000 7f946582e780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:22:52.294 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:22:52.294 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:52.294 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:22:52.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:22:52.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:52.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:22:52.524 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2535859628,v1:127.0.0.1:6827/2535859628] [v2:127.0.0.1:6828/2535859628,v1:127.0.0.1:6829/2535859628] exists,up 24692fe7-23dc-4673-87cf-40c9c8b176c7 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:22:52.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:52.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T16:22:52.529 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:22:52.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=96bbc60f-548d-4983-ba4f-f9d1403e35d0 2026-02-19T16:22:52.530 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 96bbc60f-548d-4983-ba4f-f9d1403e35d0 2026-02-19T16:22:52.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 96bbc60f-548d-4983-ba4f-f9d1403e35d0' 2026-02-19T16:22:52.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:22:52.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDcOJdpFHUKIBAAB4wUtKXGWNVqKpA+yugNKw== 2026-02-19T16:22:52.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDcOJdpFHUKIBAAB4wUtKXGWNVqKpA+yugNKw=="}' 2026-02-19T16:22:52.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 96bbc60f-548d-4983-ba4f-f9d1403e35d0 -i td/test-erasure-eio/4/new.json 2026-02-19T16:22:52.767 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:22:52.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T16:22:52.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDcOJdpFHUKIBAAB4wUtKXGWNVqKpA+yugNKw== --osd-uuid 96bbc60f-548d-4983-ba4f-f9d1403e35d0 2026-02-19T16:22:52.795 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:52.793+0000 7f20a042e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:52.796 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:52.794+0000 7f20a042e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:52.797 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:52.795+0000 7f20a042e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:52.798 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:52.796+0000 7f20a042e780 -1 bdev(0x38b8c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:22:52.798 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:52.796+0000 7f20a042e780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T16:22:53.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T16:22:53.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:22:53.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:22:53.108 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:22:53.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:22:53.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:22:53.428 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:22:53.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:22:53.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:22:53.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:22:53.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:22:53.444 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:53.443+0000 7f4dc5239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:53.446 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:53.444+0000 7f4dc5239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:53.447 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:53.445+0000 7f4dc5239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:22:53.584 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:53.582+0000 7f4dc5239780 -1 Falling back to public interface 2026-02-19T16:22:53.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:22:53.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:22:53.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:22:53.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:22:53.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:22:53.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:53.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:22:53.658 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:22:53.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:53.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:22:53.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:22:53.922 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:22:53.920+0000 7f4dc5239780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:22:54.885 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:22:54.885 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:22:54.886 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:22:54.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:22:54.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:22:54.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:22:55.117 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 36 up_thru 37 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/4149088306,v1:127.0.0.1:6835/4149088306] [v2:127.0.0.1:6836/4149088306,v1:127.0.0.1:6837/4149088306] exists,up 96bbc60f-548d-4983-ba4f-f9d1403e35d0 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:22:55.119 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:22:55.120 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:22:55.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:22:55.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:22:55.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:22:55.164 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:22:55.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:22:55.171 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:22:43.431+0000 7fd27aa42780 0 load: jerasure load: lrc 2026-02-19T16:22:55.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:22:55.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T16:22:55.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:22:55.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:22:55.832 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:22:55.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:22:56.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:22:56.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:22:56.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:22:56.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:22:56.841 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:22:56.841 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:22:56.841 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:22:56.842 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:22:56.842 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:22:56.842 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:22:56.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:22:56.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:22:56.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:22:56.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:22:56.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:22:56.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:22:57.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:22:57.154 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:22:57.154 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:22:57.155 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:22:57.155 INFO:tasks.workunit.client.0.trial127.stderr:4' 2026-02-19T16:22:57.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:22:57.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:57.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:22:57.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T16:22:57.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T16:22:57.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T16:22:57.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:57.221 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:22:57.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-19T16:22:57.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-19T16:22:57.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558' 2026-02-19T16:22:57.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:57.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:22:57.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T16:22:57.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T16:22:57.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411333' 2026-02-19T16:22:57.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:57.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:22:57.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T16:22:57.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T16:22:57.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411333 3-111669149699' 2026-02-19T16:22:57.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:22:57.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:22:57.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822659 2026-02-19T16:22:57.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822659 2026-02-19T16:22:57.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411333 3-111669149699 4-154618822659' 2026-02-19T16:22:57.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:57.481 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-19T16:22:57.481 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:57.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:22:57.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T16:22:57.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:57.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-19T16:22:57.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T16:22:57.483 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803781 2026-02-19T16:22:57.483 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:22:57.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T16:22:57.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:57.718 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-19T16:22:57.718 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:57.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:22:57.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-19T16:22:57.720 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:57.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-19T16:22:57.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-19T16:22:57.721 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607558 2026-02-19T16:22:57.721 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:22:57.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-19T16:22:57.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:57.949 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-19T16:22:57.950 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:57.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:22:57.951 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T16:22:57.951 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:57.952 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-19T16:22:57.952 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T16:22:57.952 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411333 2026-02-19T16:22:57.952 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:22:58.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T16:22:58.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:58.179 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-19T16:22:58.179 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:58.180 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:22:58.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T16:22:58.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:58.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-19T16:22:58.182 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149699 2026-02-19T16:22:58.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T16:22:58.183 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:22:58.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149697 -lt 111669149699 2026-02-19T16:22:58.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:22:59.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:22:59.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:22:59.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149700 -lt 111669149699 2026-02-19T16:22:59.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:22:59.642 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822659 2026-02-19T16:22:59.642 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:22:59.642 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:22:59.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822659 2026-02-19T16:22:59.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:22:59.644 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822659 2026-02-19T16:22:59.644 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822659' 2026-02-19T16:22:59.645 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 154618822659 2026-02-19T16:22:59.645 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:22:59.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822659 -lt 154618822659 2026-02-19T16:22:59.874 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:22:59.874 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:22:59.875 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:23:00.196 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:23:00.196 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:23:00.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:23:00.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:23:00.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:23:00.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:23:00.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:23:00.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:23:00.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:23:00.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:23:00.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:23:00.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:23:00.739 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:23:00.740 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:23:00.740 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:23:00.740 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T16:23:00.958 INFO:tasks.workunit.client.0.trial127.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T16:23:00.958 INFO:tasks.workunit.client.0.trial127.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:56.914392+0000 0'0 42:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T16:22:55.768478+0000 0'0 2026-02-19T16:22:55.768478+0000 0 0 periodic scrub scheduled @ 2026-02-20T22:24:16.530150+0000 0 0 2026-02-19T16:23:00.958 INFO:tasks.workunit.client.0.trial127.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:54.882011+0000 0'0 42:19 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-21T02:38:43.233532+0000 0 0 2026-02-19T16:23:00.958 INFO:tasks.workunit.client.0.trial127.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:54.882066+0000 0'0 42:102 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T22:25:53.911037+0000 0 0 2026-02-19T16:23:00.958 INFO:tasks.workunit.client.0.trial127.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:55.272008+0000 0'0 42:19 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:52:54.550101+0000 0 0 2026-02-19T16:23:00.958 INFO:tasks.workunit.client.0.trial127.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:50.181131+0000 0'0 43:68 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:41:49.232485+0000 0 0 2026-02-19T16:23:00.959 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-19T16:23:00.959 INFO:tasks.workunit.client.0.trial127.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T16:23:00.959 INFO:tasks.workunit.client.0.trial127.stderr:dumped pgs 2026-02-19T16:23:00.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:23:00.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:23:00.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:23:00.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T16:23:00.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:23:00.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:23:00.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:23:00.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:23:00.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:23:00.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:23:00.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:23:00.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:23:00.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:00.984 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-19T16:23:00.984 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-19T16:23:00.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-19T16:23:00.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:23:00.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T16:23:01.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-19T16:23:01.225 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-19T16:23:01.225 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:23:01.225 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:23:01.225 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:23:01.226 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:23:01.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:23:01.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:23:01.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:23:01.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:23:01.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:23:01.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:23:01.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:23:01.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-19T16:23:01.787 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already down. 2026-02-19T16:23:01.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-19T16:23:02.076 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already out. 2026-02-19T16:23:02.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T16:23:02.299 INFO:tasks.workunit.client.0.trial127.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T16:23:02.300 INFO:tasks.workunit.client.0.trial127.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:56.914392+0000 0'0 42:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T16:22:55.768478+0000 0'0 2026-02-19T16:22:55.768478+0000 0 0 periodic scrub scheduled @ 2026-02-20T22:24:16.530150+0000 0 0 2026-02-19T16:23:02.300 INFO:tasks.workunit.client.0.trial127.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:54.882011+0000 0'0 43:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-21T02:38:43.233532+0000 0 0 2026-02-19T16:23:02.300 INFO:tasks.workunit.client.0.trial127.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:54.882066+0000 0'0 43:104 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T22:25:53.911037+0000 0 0 2026-02-19T16:23:02.300 INFO:tasks.workunit.client.0.trial127.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:55.272008+0000 0'0 43:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:52:54.550101+0000 0 0 2026-02-19T16:23:02.300 INFO:tasks.workunit.client.0.trial127.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:50.181131+0000 0'0 43:68 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:41:49.232485+0000 0 0 2026-02-19T16:23:02.300 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-19T16:23:02.300 INFO:tasks.workunit.client.0.trial127.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T16:23:02.301 INFO:tasks.workunit.client.0.trial127.stderr:dumped pgs 2026-02-19T16:23:02.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-19T16:23:02.308 INFO:tasks.workunit.client.0.trial127.stderr:4+0 records in 2026-02-19T16:23:02.309 INFO:tasks.workunit.client.0.trial127.stderr:4+0 records out 2026-02-19T16:23:02.309 INFO:tasks.workunit.client.0.trial127.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 7.4799e-05 s, 54.8 MB/s 2026-02-19T16:23:02.309 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-19T16:23:02.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.741 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.741 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:02.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:02.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.101 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.101 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.298 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.298 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:03.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:03.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.834 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.834 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:04.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:04.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.359 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.359 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.460 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.982 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:05.982 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:05.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.099 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.150 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.150 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.316 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.316 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.866 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.866 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:06.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:06.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.101 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.101 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.153 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.153 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:07.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:23:07.341 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-19T16:23:07.342 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:23:07.342 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-19T16:23:07.342 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-19T16:23:07.342 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:23:07.570 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:23:07.570 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:23:07.571 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:23:07.571 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:23:07.571 INFO:tasks.workunit.client.0.trial127.stderr:2147483647' 2026-02-19T16:23:07.571 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T16:23:07.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T16:23:07.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:23:07.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-19T16:23:07.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:23:07.572 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T16:23:07.574 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:23:07.575 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:23:07.575 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:23:07.575 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:07.575 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:07.575 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:23:07.575 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:23:07.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:23:07.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:23:07.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:23:07.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-19T16:23:07.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:23:07.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:23:07.626 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:23:07.626 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:07.626 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:07.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:23:07.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:23:07.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-19T16:23:07.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:23:07.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:23:07.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-19T16:23:07.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:23:07.675 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-19T16:23:07.675 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:23:07.675 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-19T16:23:07.675 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-19T16:23:07.675 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:23:07.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:23:07.909 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:23:07.909 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:23:07.909 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:23:07.909 INFO:tasks.workunit.client.0.trial127.stderr:2147483647' 2026-02-19T16:23:07.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T16:23:07.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T16:23:07.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:23:07.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-19T16:23:07.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:23:07.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:23:07.912 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T16:23:07.913 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:23:07.913 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:23:07.913 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:23:07.913 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:07.913 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:07.913 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:23:07.913 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:23:07.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:23:07.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:23:07.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:23:07.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-19T16:23:07.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:23:07.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:23:07.961 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:23:07.961 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:07.962 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:07.962 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:23:07.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:23:07.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-19T16:23:08.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-19T16:23:08.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:23:08.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:23:08.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:23:08.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:23:08.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:23:08.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:08.012 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:23:08.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:23:08.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:23:08.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:23:08.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:23:08.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:23:08.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:23:08.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:23:08.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:23:08.015 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:23:08.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:23:08.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:23:08.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:23:08.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:23:08.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:23:08.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:23:08.029 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:08.027+0000 7f880ee0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:08.030 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:08.028+0000 7f880ee0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:08.031 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:08.029+0000 7f880ee0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:08.150 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:08.147+0000 7f880ee0b780 -1 Falling back to public interface 2026-02-19T16:23:08.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:23:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:23:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:23:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:23:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:23:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:08.243 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:23:08.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:23:08.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:08.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:23:08.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:23:08.507 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:08.506+0000 7f880ee0b780 -1 osd.2 43 log_to_monitors true 2026-02-19T16:23:09.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:23:09.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:09.470 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:23:09.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:23:09.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:09.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:23:09.694 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up out weight 0 up_from 49 up_thru 20 down_at 44 last_clean_interval [18,43) [v2:127.0.0.1:6818/2270789471,v1:127.0.0.1:6819/2270789471] [v2:127.0.0.1:6820/2270789471,v1:127.0.0.1:6821/2270789471] exists,up d07406bc-20a1-447d-bd85-5f8053335109 2026-02-19T16:23:09.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:23:09.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:23:09.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:23:09.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-19T16:23:09.946 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already in. 2026-02-19T16:23:09.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-19T16:23:24.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-19T16:23:24.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-19T16:23:24.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-19T16:23:24.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-19T16:23:24.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-19T16:23:24.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-19T16:23:24.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-19T16:23:25.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-19T16:23:25.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-19T16:23:25.187 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-19T16:23:25.188 INFO:tasks.workunit.client.0.trial127.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-19T16:23:25.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-19T16:23:25.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-19T16:23:25.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T16:23:25.406 INFO:tasks.workunit.client.0.trial127.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T16:23:25.407 INFO:tasks.workunit.client.0.trial127.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-19T16:23:10.361241+0000 47'301 56:908 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-19T16:22:55.768478+0000 0'0 2026-02-19T16:22:55.768478+0000 0 0 no scrub is scheduled 0 0 2026-02-19T16:23:25.407 INFO:tasks.workunit.client.0.trial127.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:23:09.982764+0000 0'0 56:64 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:41:19.569580+0000 0 0 2026-02-19T16:23:25.407 INFO:tasks.workunit.client.0.trial127.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:54.882066+0000 0'0 56:128 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T22:25:53.911037+0000 0 0 2026-02-19T16:23:25.407 INFO:tasks.workunit.client.0.trial127.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:55.272008+0000 0'0 56:46 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:52:54.550101+0000 0 0 2026-02-19T16:23:25.407 INFO:tasks.workunit.client.0.trial127.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:23:09.982837+0000 0'0 56:111 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T21:27:11.485321+0000 0 0 2026-02-19T16:23:25.408 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-19T16:23:25.408 INFO:tasks.workunit.client.0.trial127.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T16:23:25.408 INFO:tasks.workunit.client.0.trial127.stderr:dumped pgs 2026-02-19T16:23:25.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:23:25.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:23:25.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:23:25.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:23:25.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:23:25.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:23:25.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:23:25.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-19T16:23:30.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-19T16:23:25.571686+0000 47'301 58:920 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-19T16:22:55.768478+0000 0'0 2026-02-19T16:22:55.768478+0000 0 0 no scrub is scheduled 0 0 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:23:09.982764+0000 0'0 56:64 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:41:19.569580+0000 0 0 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:54.882066+0000 0'0 56:128 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T22:25:53.911037+0000 0 0 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:22:55.272008+0000 0'0 56:46 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:52:54.550101+0000 0 0 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-19T16:23:25.570148+0000 0'0 58:119 [1,0] 1 [1,0] 1 0'0 2026-02-19T16:22:41.442970+0000 0'0 2026-02-19T16:22:41.442970+0000 0 0 periodic scrub scheduled @ 2026-02-20T18:09:42.985169+0000 0 0 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-19T16:23:30.740 INFO:tasks.workunit.client.0.trial127.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T16:23:30.741 INFO:tasks.workunit.client.0.trial127.stderr:dumped pgs 2026-02-19T16:23:30.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:23:30.808 INFO:tasks.workunit.client.0.trial127.stdout:{ 2026-02-19T16:23:30.809 INFO:tasks.workunit.client.0.trial127.stdout: "num_missing": 1, 2026-02-19T16:23:30.809 INFO:tasks.workunit.client.0.trial127.stdout: "num_unfound": 1, 2026-02-19T16:23:30.809 INFO:tasks.workunit.client.0.trial127.stdout: "objects": [ 2026-02-19T16:23:30.809 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.809 INFO:tasks.workunit.client.0.trial127.stdout: "oid": { 2026-02-19T16:23:30.809 INFO:tasks.workunit.client.0.trial127.stdout: "oid": "obj250", 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "key": "", 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "snapid": -2, 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "hash": 2249616407, 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "max": 0, 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "pool": 2, 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "namespace": "" 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "need": "47'251", 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "have": "0'0", 2026-02-19T16:23:30.810 INFO:tasks.workunit.client.0.trial127.stdout: "flags": "none", 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: "locations": [ 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: ] 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: "state": "NotRecovering", 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: "available_might_have_unfound": true, 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: "might_have_unfound": [ 2026-02-19T16:23:30.811 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.812 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "2(4)", 2026-02-19T16:23:30.812 INFO:tasks.workunit.client.0.trial127.stdout: "status": "osd is down" 2026-02-19T16:23:30.812 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.812 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.812 INFO:tasks.workunit.client.0.trial127.stdout: "more": false 2026-02-19T16:23:30.812 INFO:tasks.workunit.client.0.trial127.stdout:} 2026-02-19T16:23:30.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout:{ 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "snap_trimq": "[]", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "snap_trimq_len": 0, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "epoch": 58, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "acting_recovery_backfill": [ 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "info": { 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s0", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "0", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'301", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'250", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "47'200", 2026-02-19T16:23:30.872 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 301, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "57", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 57, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 57, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 57, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'301", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 921, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 58, 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:23:25.571686+0000", 2026-02-19T16:23:30.873 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:23:25.571686+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:23:25.571686+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:23:25.571686+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:23:00.980942+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:23:25.570302+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:23:25.570302+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:23:25.571686+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:23:25.569079+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:23:25.568978+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 57, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "47'200", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "47'200", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 200, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:23:30.874 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 1232896, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 301, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 1505, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 1, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 1, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 303, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 1, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 301, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 301, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 1204, 2026-02-19T16:23:30.875 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:23:30.876 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 300 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),4(2)", 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 1 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.877 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "peer_info": [ 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "peer": "0(3)", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s3", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "3", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'301", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'301", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "47'200", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 301, 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:23:30.878 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 57, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 57, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 57, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'301", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 911, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 57, 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+remapped", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.879 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:23:00.980942+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:23:01.600526+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:23:01.600471+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 57, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "47'200", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "47'200", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 200, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:23:30.880 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T20:12:25.344069+0000", 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 1232896, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 301, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 1505, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 301, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 301, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 301, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 1204, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:23:30.881 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.882 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 301 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:23:30.883 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "peer": "1(1)", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s1", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "1", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'301", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'301", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "47'200", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 301, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 57, 2026-02-19T16:23:30.884 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 57, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 57, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'301", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 331, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 47, 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+remapped", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:23:00.980942+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.885 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:23:01.600526+0000", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:23:01.600471+0000", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 57, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "47'200", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "47'200", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 200, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:23:30.886 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T20:12:25.344069+0000", 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 1232896, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 301, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 1505, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 1, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 301, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 301, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 301, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 1204, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:23:30.887 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.888 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 301 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "peer": "4(2)", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s2", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "2", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'301", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'301", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "47'200", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 301, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.889 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 57, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 57, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 57, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'301", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 331, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 47, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+remapped", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:23:00.980942+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:23:01.601801+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:23:07.337649+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:23:01.600526+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:23:01.600471+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 57, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "47'200", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "47'200", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:22:55.768478+0000", 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 200, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:23:30.890 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T20:12:25.344069+0000", 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 1232896, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 301, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 1505, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 301, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 301, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 301, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 1204, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:23:30.891 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: 2147483647 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 301 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 58, 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.892 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "recovery_state": [ 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "name": "Started/Primary/Active", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "enter_time": "2026-02-19T16:23:25.568993+0000", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "might_have_unfound": [ 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "0(3)", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "status": "already probed" 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "1(1)", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "status": "already probed" 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "2(4)", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "status": "osd is down" 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "4(2)", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "status": "already probed" 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "recovery_progress": { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "backfill_targets": [], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "waiting_on_backfill": [], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill_started": "MIN", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "backfill_info": { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "begin": "MIN", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "end": "MIN", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "objects": [] 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "peer_backfill_info": [], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "backfills_in_flight": [], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "recovering": [], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "pg_backend": { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "recovery_ops": [], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "read_ops": [] 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "name": "Started", 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "enter_time": "2026-02-19T16:23:25.468715+0000" 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "scrubber": { 2026-02-19T16:23:30.893 INFO:tasks.workunit.client.0.trial127.stdout: "active": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "must_scrub": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "must_deep_scrub": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "must_repair": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "sched_time": "2026-02-20T22:56:57.741702+0000", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "schedule": "not registered for scrubbing", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "urgency": "periodic-regular", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "shallow-target": { 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s0", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "level": "shallow", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "urgency": "periodic-regular", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "sched_time": "2026-02-20T22:56:57.741702+0000", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "orig_sched_time": "2026-02-20T22:56:57.741702+0000", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "last_issue": "ok", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "forced": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "eligible": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "queued": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "active": false 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "deep-target": { 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s0", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "level": "deep", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "urgency": "periodic-regular", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "sched_time": "2026-02-27T08:56:03.698573+0000", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "orig_sched_time": "2026-02-27T08:56:03.698573+0000", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "last_issue": "ok", 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "forced": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "eligible": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "queued": false, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "active": false 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout: "agent_state": {} 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stdout:} 2026-02-19T16:23:30.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:23:30.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-19T16:23:30.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:23:30.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-19T16:23:31.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-19T16:23:31.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-19T16:23:31.010 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:23:31.010 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-19T16:23:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-19T16:23:31.074 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-19T16:23:31.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-19T16:23:31.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:23:31.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-19T16:23:31.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:23:31.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:31.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:23:31.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:23:31.145 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:23:31.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:23:31.146 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:23:31.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:23:31.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:23:31.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:23:31.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:23:31.157 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:31.154+0000 7ffb64e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:31.159 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:31.156+0000 7ffb64e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:31.160 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:31.157+0000 7ffb64e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:31.295 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:31.291+0000 7ffb64e37780 -1 Falling back to public interface 2026-02-19T16:23:31.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:23:31.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:23:31.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:23:31.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:23:31.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:23:31.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:31.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:23:31.372 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:23:31.373 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:31.373 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:23:31.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:23:31.674 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:31.671+0000 7ffb64e37780 -1 osd.2 56 log_to_monitors true 2026-02-19T16:23:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:23:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:23:32.601 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:23:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:23:32.832 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 60 up_thru 20 down_at 57 last_clean_interval [49,56) [v2:127.0.0.1:6818/3576666927,v1:127.0.0.1:6819/3576666927] [v2:127.0.0.1:6820/3576666927,v1:127.0.0.1:6821/3576666927] exists,up d07406bc-20a1-447d-bd85-5f8053335109 2026-02-19T16:23:32.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:23:32.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:23:32.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:23:32.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-19T16:23:37.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-19T16:23:37.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-19T16:23:37.900 INFO:tasks.workunit.client.0.trial127.stderr:pg has no unfound objects 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:23:37.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:23:37.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:23:37.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:23:37.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:23:37.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:23:37.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:23:37.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:23:37.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:23:37.992 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:23:38.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:23:38.219 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:23:38.219 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:23:38.219 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:23:38.219 INFO:tasks.workunit.client.0.trial127.stderr:4' 2026-02-19T16:23:38.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:23:38.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:23:38.220 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:23:38.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-19T16:23:38.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-19T16:23:38.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-19T16:23:38.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:23:38.283 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:23:38.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607568 2026-02-19T16:23:38.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607568 2026-02-19T16:23:38.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607568' 2026-02-19T16:23:38.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:23:38.351 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:23:38.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037764 2026-02-19T16:23:38.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037764 2026-02-19T16:23:38.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607568 2-257698037764' 2026-02-19T16:23:38.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:23:38.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:23:38.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149710 2026-02-19T16:23:38.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149710 2026-02-19T16:23:38.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607568 2-257698037764 3-111669149710' 2026-02-19T16:23:38.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:23:38.481 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:23:38.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822669 2026-02-19T16:23:38.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822669 2026-02-19T16:23:38.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607568 2-257698037764 3-111669149710 4-154618822669' 2026-02-19T16:23:38.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:23:38.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-19T16:23:38.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:23:38.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:23:38.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-19T16:23:38.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:23:38.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-19T16:23:38.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-19T16:23:38.548 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803791 2026-02-19T16:23:38.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:23:38.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-19T16:23:38.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:23:39.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:23:39.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:23:40.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-19T16:23:40.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:23:40.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607568 2026-02-19T16:23:40.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:23:40.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:23:40.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607568 2026-02-19T16:23:40.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:23:40.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607568 2026-02-19T16:23:40.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607568' 2026-02-19T16:23:40.006 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607568 2026-02-19T16:23:40.006 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:23:40.230 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607568 -lt 51539607568 2026-02-19T16:23:40.230 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:23:40.231 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-257698037764 2026-02-19T16:23:40.231 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:23:40.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:23:40.232 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-257698037764 2026-02-19T16:23:40.232 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:23:40.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=257698037764 2026-02-19T16:23:40.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 257698037764' 2026-02-19T16:23:40.234 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 257698037764 2026-02-19T16:23:40.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:23:40.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037764 -lt 257698037764 2026-02-19T16:23:40.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:23:40.464 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149710 2026-02-19T16:23:40.464 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:23:40.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:23:40.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149710 2026-02-19T16:23:40.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:23:40.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149710 2026-02-19T16:23:40.468 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149710 2026-02-19T16:23:40.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149710' 2026-02-19T16:23:40.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:23:40.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149710 -lt 111669149710 2026-02-19T16:23:40.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:23:40.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822669 2026-02-19T16:23:40.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:23:40.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:23:40.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822669 2026-02-19T16:23:40.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:23:40.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822669 2026-02-19T16:23:40.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822669' 2026-02-19T16:23:40.698 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 154618822669 2026-02-19T16:23:40.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:23:40.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822669 -lt 154618822669 2026-02-19T16:23:40.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:23:40.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:23:40.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:23:41.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:23:41.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:23:41.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:23:41.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:23:41.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:23:41.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:23:41.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:23:41.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:23:41.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:23:41.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:23:41.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:23:41.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:23:41.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:23:41.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:23:41.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:23:41.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-19T16:23:41.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-19T16:23:41.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-19T16:23:41.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-19T16:23:41.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-19T16:23:41.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-19T16:23:41.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-19T16:23:41.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-19T16:23:41.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-19T16:23:41.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-19T16:23:41.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-19T16:23:41.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-19T16:23:41.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.980 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-19T16:23:41.980 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-19T16:23:41.996 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-19T16:23:41.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-19T16:23:42.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.030 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.031 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.031 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-19T16:23:42.031 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-19T16:23:42.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-19T16:23:42.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-19T16:23:42.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-19T16:23:42.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-19T16:23:42.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-19T16:23:42.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-19T16:23:42.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-19T16:23:42.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.180 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-19T16:23:42.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-19T16:23:42.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-19T16:23:42.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-19T16:23:42.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-19T16:23:42.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-19T16:23:42.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-19T16:23:42.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-19T16:23:42.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.316 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-19T16:23:42.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-19T16:23:42.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.350 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-19T16:23:42.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-19T16:23:42.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-19T16:23:42.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-19T16:23:42.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-19T16:23:42.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.435 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-19T16:23:42.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-19T16:23:42.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-19T16:23:42.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-19T16:23:42.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-19T16:23:42.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-19T16:23:42.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-19T16:23:42.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-19T16:23:42.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-19T16:23:42.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-19T16:23:42.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-19T16:23:42.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-19T16:23:42.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-19T16:23:42.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.656 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-19T16:23:42.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-19T16:23:42.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-19T16:23:42.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-19T16:23:42.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.723 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-19T16:23:42.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.741 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-19T16:23:42.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-19T16:23:42.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-19T16:23:42.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-19T16:23:42.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-19T16:23:42.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-19T16:23:42.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-19T16:23:42.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-19T16:23:42.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-19T16:23:42.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.897 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-19T16:23:42.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-19T16:23:42.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-19T16:23:42.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-19T16:23:42.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-19T16:23:42.967 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-19T16:23:42.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:42.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:42.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-19T16:23:42.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-19T16:23:43.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-19T16:23:43.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-19T16:23:43.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-19T16:23:43.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-19T16:23:43.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-19T16:23:43.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-19T16:23:43.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-19T16:23:43.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-19T16:23:43.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-19T16:23:43.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-19T16:23:43.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-19T16:23:43.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-19T16:23:43.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-19T16:23:43.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-19T16:23:43.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-19T16:23:43.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-19T16:23:43.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-19T16:23:43.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-19T16:23:43.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-19T16:23:43.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-19T16:23:43.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.359 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-19T16:23:43.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-19T16:23:43.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-19T16:23:43.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-19T16:23:43.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-19T16:23:43.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-19T16:23:43.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-19T16:23:43.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-19T16:23:43.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-19T16:23:43.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-19T16:23:43.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-19T16:23:43.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-19T16:23:43.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-19T16:23:43.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-19T16:23:43.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-19T16:23:43.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-19T16:23:43.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-19T16:23:43.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-19T16:23:43.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-19T16:23:43.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-19T16:23:43.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-19T16:23:43.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-19T16:23:43.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-19T16:23:43.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-19T16:23:43.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-19T16:23:43.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-19T16:23:43.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-19T16:23:43.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-19T16:23:43.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-19T16:23:43.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.852 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.852 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-19T16:23:43.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.868 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-19T16:23:43.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.885 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-19T16:23:43.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-19T16:23:43.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-19T16:23:43.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-19T16:23:43.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-19T16:23:43.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-19T16:23:43.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-19T16:23:43.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:43.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:43.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-19T16:23:43.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-19T16:23:44.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-19T16:23:44.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-19T16:23:44.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-19T16:23:44.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-19T16:23:44.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-19T16:23:44.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-19T16:23:44.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-19T16:23:44.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-19T16:23:44.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-19T16:23:44.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-19T16:23:44.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-19T16:23:44.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-19T16:23:44.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-19T16:23:44.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-19T16:23:44.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-19T16:23:44.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-19T16:23:44.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-19T16:23:44.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-19T16:23:44.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-19T16:23:44.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-19T16:23:44.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-19T16:23:44.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-19T16:23:44.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-19T16:23:44.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-19T16:23:44.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-19T16:23:44.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-19T16:23:44.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-19T16:23:44.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-19T16:23:44.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-19T16:23:44.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-19T16:23:44.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-19T16:23:44.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-19T16:23:44.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-19T16:23:44.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-19T16:23:44.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-19T16:23:44.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-19T16:23:44.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-19T16:23:44.632 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-19T16:23:44.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-19T16:23:44.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-19T16:23:44.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-19T16:23:44.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-19T16:23:44.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-19T16:23:44.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-19T16:23:44.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-19T16:23:44.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-19T16:23:44.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-19T16:23:44.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-19T16:23:44.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-19T16:23:44.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-19T16:23:44.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-19T16:23:44.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.889 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.889 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-19T16:23:44.889 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.905 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.905 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-19T16:23:44.905 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-19T16:23:44.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-19T16:23:44.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-19T16:23:44.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-19T16:23:44.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-19T16:23:44.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:44.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:44.990 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-19T16:23:44.990 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-19T16:23:45.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-19T16:23:45.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-19T16:23:45.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-19T16:23:45.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-19T16:23:45.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-19T16:23:45.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-19T16:23:45.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-19T16:23:45.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-19T16:23:45.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-19T16:23:45.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-19T16:23:45.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-19T16:23:45.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-19T16:23:45.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-19T16:23:45.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-19T16:23:45.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-19T16:23:45.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-19T16:23:45.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.294 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.294 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-19T16:23:45.294 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-19T16:23:45.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-19T16:23:45.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-19T16:23:45.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-19T16:23:45.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-19T16:23:45.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-19T16:23:45.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-19T16:23:45.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.429 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-19T16:23:45.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-19T16:23:45.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-19T16:23:45.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-19T16:23:45.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-19T16:23:45.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-19T16:23:45.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-19T16:23:45.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-19T16:23:45.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.566 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-19T16:23:45.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-19T16:23:45.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-19T16:23:45.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.618 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-19T16:23:45.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.635 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-19T16:23:45.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.652 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.653 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.653 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-19T16:23:45.653 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-19T16:23:45.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-19T16:23:45.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-19T16:23:45.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-19T16:23:45.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-19T16:23:45.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-19T16:23:45.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-19T16:23:45.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-19T16:23:45.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-19T16:23:45.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-19T16:23:45.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-19T16:23:45.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-19T16:23:45.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-19T16:23:45.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.885 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-19T16:23:45.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-19T16:23:45.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-19T16:23:45.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-19T16:23:45.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-19T16:23:45.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-19T16:23:45.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-19T16:23:45.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:45.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:45.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-19T16:23:45.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-19T16:23:46.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-19T16:23:46.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.036 INFO:tasks.workunit.client.0.trial127.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-19T16:23:46.037 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.037 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-19T16:23:46.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-19T16:23:46.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.071 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-19T16:23:46.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-19T16:23:46.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-19T16:23:46.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-19T16:23:46.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-19T16:23:46.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-19T16:23:46.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-19T16:23:46.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-19T16:23:46.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-19T16:23:46.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-19T16:23:46.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-19T16:23:46.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-19T16:23:46.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-19T16:23:46.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-19T16:23:46.298 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-19T16:23:46.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-19T16:23:46.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-19T16:23:46.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.363 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.363 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-19T16:23:46.363 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-19T16:23:46.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-19T16:23:46.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-19T16:23:46.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-19T16:23:46.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-19T16:23:46.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-19T16:23:46.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-19T16:23:46.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-19T16:23:46.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-19T16:23:46.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-19T16:23:46.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-19T16:23:46.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-19T16:23:46.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-19T16:23:46.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-19T16:23:46.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-19T16:23:46.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-19T16:23:46.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-19T16:23:46.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-19T16:23:46.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-19T16:23:46.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-19T16:23:46.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-19T16:23:46.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-19T16:23:46.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.748 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-19T16:23:46.749 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-19T16:23:46.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-19T16:23:46.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-19T16:23:46.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-19T16:23:46.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-19T16:23:46.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-19T16:23:46.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.866 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:23:46.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-19T16:23:46.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-19T16:23:46.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:23:46.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:23:46.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:23:46.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:23:47.156 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:23:47.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:23:47.447 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:23:47.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:23:47.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:23:47.566 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:23:47.566 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:23:47.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:23:47.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:23:47.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:23:47.568 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:23:47.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:23:47.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:23:47.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:23:47.569 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:23:47.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:23:47.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:23:47.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:23:47.611 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:23:47.611 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:47.611 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:47.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:23:47.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:23:47.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:23:47.614 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:23:47.614 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:23:47.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:23:47.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:23:47.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:23:47.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:23:47.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:23:47.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:23:47.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:23:47.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:23:47.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:23:47.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:23:47.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:23:47.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:23:47.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:23:47.618 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:23:47.618 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:23:47.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:23:47.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:23:47.620 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:23:47.620 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:47.621 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:47.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:23:47.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:23:47.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:23:47.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:23:47.622 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:23:47.622 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:47.623 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:47.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:23:47.623 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:23:47.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:23:47.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:47.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:23:47.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:23:47.671 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:23:47.671 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:23:47.672 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:23:47.672 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:23:47.672 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:23:47.672 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:23:47.672 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:23:47.672 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:23:47.672 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:23:47.673 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:47.673 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:47.673 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:23:47.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:23:47.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:23:47.715 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:23:47.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:23:47.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:23:47.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:23:47.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:23:47.716 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:23:47.716 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:23:47.717 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:23:47.717 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:23:47.717 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:47.717 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:47.718 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:23:47.718 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:23:47.718 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:23:47.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:23:47.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:23:47.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:23:47.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:23:47.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:23:47.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:23:47.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:47.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:23:47.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:23:47.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:23:47.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:23:47.967 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:23:47.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:23:48.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:23:48.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:23:49.014 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:23:49.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:23:49.020 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:23:47.658+0000 7f7089435c00 0 load: jerasure load: lrc 2026-02-19T16:23:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-19T16:23:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-19T16:23:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-19T16:23:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-19T16:23:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T16:23:49.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:23:49.021 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T16:23:49.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T16:23:49.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:23:49.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:23:49.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:23:49.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:23:49.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:23:49.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:23:49.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:23:49.028 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:23:49.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0ff71ec1-0c7c-4218-b149-c8c6ac3b066f 2026-02-19T16:23:49.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0ff71ec1-0c7c-4218-b149-c8c6ac3b066f' 2026-02-19T16:23:49.029 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 0ff71ec1-0c7c-4218-b149-c8c6ac3b066f 2026-02-19T16:23:49.029 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:23:49.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAVOZdpG7IHAhAAEHKSC7ilkKhG1a3LG1BGpQ== 2026-02-19T16:23:49.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAVOZdpG7IHAhAAEHKSC7ilkKhG1a3LG1BGpQ=="}' 2026-02-19T16:23:49.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0ff71ec1-0c7c-4218-b149-c8c6ac3b066f -i td/test-erasure-eio/0/new.json 2026-02-19T16:23:49.141 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:23:49.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:23:49.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAVOZdpG7IHAhAAEHKSC7ilkKhG1a3LG1BGpQ== --osd-uuid 0ff71ec1-0c7c-4218-b149-c8c6ac3b066f 2026-02-19T16:23:49.164 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.163+0000 7fe39b233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:49.166 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.164+0000 7fe39b233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:49.167 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.165+0000 7fe39b233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:49.167 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.165+0000 7fe39b233780 -1 bdev(0x2882c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:23:49.167 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.166+0000 7fe39b233780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:23:49.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:23:49.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:23:49.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:23:49.581 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:23:49.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:23:49.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:23:49.683 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:23:49.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:23:49.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:23:49.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:23:49.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:23:49.698 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.696+0000 7f93e842b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:49.700 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.698+0000 7f93e842b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:49.701 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.699+0000 7f93e842b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:49.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:23:49.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:23:49.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:23:49.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:23:49.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:23:49.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:49.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:23:49.782 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:23:49.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:49.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:23:49.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:23:49.933 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:49.931+0000 7f93e842b780 -1 Falling back to public interface 2026-02-19T16:23:50.192 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:50.190+0000 7f93e842b780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:23:50.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:23:50.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:50.880 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:23:50.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:23:50.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:50.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:23:51.112 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3367215283,v1:127.0.0.1:6803/3367215283] [v2:127.0.0.1:6804/3367215283,v1:127.0.0.1:6805/3367215283] exists,up 0ff71ec1-0c7c-4218-b149-c8c6ac3b066f 2026-02-19T16:23:51.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:23:51.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:23:51.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:23:51.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:51.119 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:23:51.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=24112f1c-190f-4145-a11b-a3c92d669def 2026-02-19T16:23:51.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 24112f1c-190f-4145-a11b-a3c92d669def' 2026-02-19T16:23:51.122 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 24112f1c-190f-4145-a11b-a3c92d669def 2026-02-19T16:23:51.122 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:23:51.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAXOZdpmUJtBxAAWGOrjXnzx2/cIlDd4hvljw== 2026-02-19T16:23:51.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAXOZdpmUJtBxAAWGOrjXnzx2/cIlDd4hvljw=="}' 2026-02-19T16:23:51.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 24112f1c-190f-4145-a11b-a3c92d669def -i td/test-erasure-eio/1/new.json 2026-02-19T16:23:51.359 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:23:51.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:23:51.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAXOZdpmUJtBxAAWGOrjXnzx2/cIlDd4hvljw== --osd-uuid 24112f1c-190f-4145-a11b-a3c92d669def 2026-02-19T16:23:51.385 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:51.383+0000 7f5668a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:51.386 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:51.384+0000 7f5668a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:51.387 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:51.385+0000 7f5668a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:51.387 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:51.385+0000 7f5668a29780 -1 bdev(0x3bb0c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:23:51.388 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:51.385+0000 7f5668a29780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:23:51.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:23:51.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:23:51.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:23:51.706 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:23:51.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:23:52.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:23:52.024 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:23:52.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:23:52.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:23:52.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:23:52.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:23:52.038 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:52.036+0000 7f7a7fc1f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:52.039 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:52.038+0000 7f7a7fc1f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:52.040 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:52.039+0000 7f7a7fc1f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:52.164 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:52.162+0000 7f7a7fc1f780 -1 Falling back to public interface 2026-02-19T16:23:52.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:23:52.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:23:52.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:23:52.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:23:52.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:23:52.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:52.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:23:52.252 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:23:52.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:52.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:23:52.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:23:52.512 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:52.510+0000 7f7a7fc1f780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:23:53.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:23:53.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:53.482 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:23:53.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:23:53.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:53.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:23:53.711 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2827073732,v1:127.0.0.1:6811/2827073732] [v2:127.0.0.1:6812/2827073732,v1:127.0.0.1:6813/2827073732] exists,up 24112f1c-190f-4145-a11b-a3c92d669def 2026-02-19T16:23:53.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:23:53.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:23:53.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:23:53.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:53.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:23:53.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:23:53.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:23:53.718 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:23:53.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=40cbe0c5-ac14-4c7a-8991-7f4312651c71 2026-02-19T16:23:53.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 40cbe0c5-ac14-4c7a-8991-7f4312651c71' 2026-02-19T16:23:53.719 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 40cbe0c5-ac14-4c7a-8991-7f4312651c71 2026-02-19T16:23:53.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:23:53.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAZOZdpxgAfKxAAUXsYF2520/bmwwOIPiZmgw== 2026-02-19T16:23:53.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAZOZdpxgAfKxAAUXsYF2520/bmwwOIPiZmgw=="}' 2026-02-19T16:23:53.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 40cbe0c5-ac14-4c7a-8991-7f4312651c71 -i td/test-erasure-eio/2/new.json 2026-02-19T16:23:53.956 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:23:53.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:23:53.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAZOZdpxgAfKxAAUXsYF2520/bmwwOIPiZmgw== --osd-uuid 40cbe0c5-ac14-4c7a-8991-7f4312651c71 2026-02-19T16:23:53.981 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:53.980+0000 7f6aeb60a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:53.983 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:53.981+0000 7f6aeb60a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:53.984 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:53.982+0000 7f6aeb60a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:53.984 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:53.982+0000 7f6aeb60a780 -1 bdev(0x38b2c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:23:53.984 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:53.982+0000 7f6aeb60a780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:23:54.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:23:54.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:23:54.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:23:54.312 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:23:54.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:23:54.626 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:23:54.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:23:54.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:23:54.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:23:54.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:23:54.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:23:54.642 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:54.640+0000 7f5d12e2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:54.643 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:54.642+0000 7f5d12e2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:54.644 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:54.642+0000 7f5d12e2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:54.762 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:54.760+0000 7f5d12e2d780 -1 Falling back to public interface 2026-02-19T16:23:54.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:23:54.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:23:54.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:23:54.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:23:54.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:23:54.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:54.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:23:54.856 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:23:54.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:54.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:23:55.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:23:55.142 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:55.140+0000 7f5d12e2d780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:23:56.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:23:56.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:56.083 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:23:56.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:23:56.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:56.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:23:56.310 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3873852383,v1:127.0.0.1:6819/3873852383] [v2:127.0.0.1:6820/3873852383,v1:127.0.0.1:6821/3873852383] exists,up 40cbe0c5-ac14-4c7a-8991-7f4312651c71 2026-02-19T16:23:56.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:23:56.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:23:56.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:23:56.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:23:56.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:56.311 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:23:56.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:23:56.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:23:56.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b4872867-e058-4969-af84-dc7d9d56ec66 2026-02-19T16:23:56.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 b4872867-e058-4969-af84-dc7d9d56ec66' 2026-02-19T16:23:56.314 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 b4872867-e058-4969-af84-dc7d9d56ec66 2026-02-19T16:23:56.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:23:56.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAcOZdpLtcvExAAI/VFcR1ereR/qKYhAENZyw== 2026-02-19T16:23:56.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAcOZdpLtcvExAAI/VFcR1ereR/qKYhAENZyw=="}' 2026-02-19T16:23:56.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b4872867-e058-4969-af84-dc7d9d56ec66 -i td/test-erasure-eio/3/new.json 2026-02-19T16:23:56.556 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:23:56.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:23:56.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAcOZdpLtcvExAAI/VFcR1ereR/qKYhAENZyw== --osd-uuid b4872867-e058-4969-af84-dc7d9d56ec66 2026-02-19T16:23:56.581 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:56.579+0000 7f73bcdfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:56.582 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:56.580+0000 7f73bcdfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:56.583 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:56.581+0000 7f73bcdfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:56.583 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:56.581+0000 7f73bcdfd780 -1 bdev(0x3f96c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:23:56.583 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:56.581+0000 7f73bcdfd780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:23:56.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:23:56.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:23:56.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:23:56.912 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:23:56.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:23:57.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:23:57.234 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:23:57.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:23:57.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:23:57.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:23:57.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:23:57.248 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:57.247+0000 7f9a9040f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:57.250 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:57.248+0000 7f9a9040f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:57.251 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:57.249+0000 7f9a9040f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:57.380 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:57.378+0000 7f9a9040f780 -1 Falling back to public interface 2026-02-19T16:23:57.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:23:57.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:23:57.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:23:57.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:23:57.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:23:57.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:57.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:23:57.463 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:23:57.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:57.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:23:57.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:23:57.708 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:57.706+0000 7f9a9040f780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:23:58.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:23:58.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:23:58.691 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:23:58.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:23:58.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:23:58.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:23:58.921 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/195271158,v1:127.0.0.1:6827/195271158] [v2:127.0.0.1:6828/195271158,v1:127.0.0.1:6829/195271158] exists,up b4872867-e058-4969-af84-dc7d9d56ec66 2026-02-19T16:23:58.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:23:58.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:23:58.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:23:58.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:23:58.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T16:23:58.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:23:58.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=29daebee-f6a0-4664-9dea-fc8199dffaa1 2026-02-19T16:23:58.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 29daebee-f6a0-4664-9dea-fc8199dffaa1' 2026-02-19T16:23:58.929 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 29daebee-f6a0-4664-9dea-fc8199dffaa1 2026-02-19T16:23:58.929 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:23:58.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAeOZdpjG+iNxAAuf6IYVdmHk/NibReLN7uaw== 2026-02-19T16:23:58.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAeOZdpjG+iNxAAuf6IYVdmHk/NibReLN7uaw=="}' 2026-02-19T16:23:58.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 29daebee-f6a0-4664-9dea-fc8199dffaa1 -i td/test-erasure-eio/4/new.json 2026-02-19T16:23:59.169 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:23:59.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T16:23:59.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAeOZdpjG+iNxAAuf6IYVdmHk/NibReLN7uaw== --osd-uuid 29daebee-f6a0-4664-9dea-fc8199dffaa1 2026-02-19T16:23:59.194 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.193+0000 7f03f923b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:59.196 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.194+0000 7f03f923b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:59.197 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.195+0000 7f03f923b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:59.197 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.196+0000 7f03f923b780 -1 bdev(0x3536c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:23:59.198 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.196+0000 7f03f923b780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T16:23:59.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T16:23:59.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:23:59.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:23:59.517 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:23:59.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:23:59.850 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:23:59.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:23:59.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:23:59.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:23:59.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:23:59.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:23:59.866 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.864+0000 7f1b93638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:59.867 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.865+0000 7f1b93638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:23:59.868 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:23:59.866+0000 7f1b93638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:00.036 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:00.035+0000 7f1b93638780 -1 Falling back to public interface 2026-02-19T16:24:00.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:24:00.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:00.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:24:00.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:00.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:00.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:00.087 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:00.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:00.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:00.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:24:00.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:00.339 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:00.337+0000 7f1b93638780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:24:01.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:01.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:01.314 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:01.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:01.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:01.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:24:01.542 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3589201425,v1:127.0.0.1:6835/3589201425] [v2:127.0.0.1:6836/3589201425,v1:127.0.0.1:6837/3589201425] exists,up 29daebee-f6a0-4664-9dea-fc8199dffaa1 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:01.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:01.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T16:24:01.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:01.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=87b06001-f9d8-445f-9d4c-144ffa1bf569 2026-02-19T16:24:01.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 87b06001-f9d8-445f-9d4c-144ffa1bf569' 2026-02-19T16:24:01.547 INFO:tasks.workunit.client.0.trial127.stdout:add osd5 87b06001-f9d8-445f-9d4c-144ffa1bf569 2026-02-19T16:24:01.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:01.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAhOZdpm7UdIRAA96NEpyId/eoA0J5Lij+syg== 2026-02-19T16:24:01.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAhOZdpm7UdIRAA96NEpyId/eoA0J5Lij+syg=="}' 2026-02-19T16:24:01.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 87b06001-f9d8-445f-9d4c-144ffa1bf569 -i td/test-erasure-eio/5/new.json 2026-02-19T16:24:01.788 INFO:tasks.workunit.client.0.trial127.stdout:5 2026-02-19T16:24:01.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T16:24:01.797 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAhOZdpm7UdIRAA96NEpyId/eoA0J5Lij+syg== --osd-uuid 87b06001-f9d8-445f-9d4c-144ffa1bf569 2026-02-19T16:24:01.813 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:01.811+0000 7f2fb944b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:01.814 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:01.812+0000 7f2fb944b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:01.815 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:01.813+0000 7f2fb944b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:01.815 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:01.814+0000 7f2fb944b780 -1 bdev(0x41acc00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:01.816 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:01.814+0000 7f2fb944b780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T16:24:02.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T16:24:02.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:02.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T16:24:02.169 INFO:tasks.workunit.client.0.trial127.stdout:adding osd5 key to auth repository 2026-02-19T16:24:02.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:02.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T16:24:02.483 INFO:tasks.workunit.client.0.trial127.stdout:start osd.5 2026-02-19T16:24:02.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:02.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:02.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:02.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:02.497 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:02.494+0000 7f935b01f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:02.498 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:02.496+0000 7f935b01f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:02.499 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:02.497+0000 7f935b01f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:02.627 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:02.624+0000 7f935b01f780 -1 Falling back to public interface 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:02.710 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:02.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:02.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:24:02.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:02.972 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:02.969+0000 7f935b01f780 -1 osd.5 0 log_to_monitors true 2026-02-19T16:24:03.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:03.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:03.942 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:03.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:03.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:03.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:24:04.171 INFO:tasks.workunit.client.0.trial127.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1465651214,v1:127.0.0.1:6843/1465651214] [v2:127.0.0.1:6844/1465651214,v1:127.0.0.1:6845/1465651214] exists,up 87b06001-f9d8-445f-9d4c-144ffa1bf569 2026-02-19T16:24:04.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T16:24:04.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:04.176 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:04.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=66274346-d2d5-4ac0-bde0-c4a046d9f44e 2026-02-19T16:24:04.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 66274346-d2d5-4ac0-bde0-c4a046d9f44e' 2026-02-19T16:24:04.179 INFO:tasks.workunit.client.0.trial127.stdout:add osd6 66274346-d2d5-4ac0-bde0-c4a046d9f44e 2026-02-19T16:24:04.179 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:04.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAkOZdpwZXrChAAecvTZh5ikGYTbqI9MgS+/A== 2026-02-19T16:24:04.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAkOZdpwZXrChAAecvTZh5ikGYTbqI9MgS+/A=="}' 2026-02-19T16:24:04.185 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 66274346-d2d5-4ac0-bde0-c4a046d9f44e -i td/test-erasure-eio/6/new.json 2026-02-19T16:24:04.417 INFO:tasks.workunit.client.0.trial127.stdout:6 2026-02-19T16:24:04.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T16:24:04.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAkOZdpwZXrChAAecvTZh5ikGYTbqI9MgS+/A== --osd-uuid 66274346-d2d5-4ac0-bde0-c4a046d9f44e 2026-02-19T16:24:04.441 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:04.439+0000 7f0cefc1c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:04.443 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:04.440+0000 7f0cefc1c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:04.444 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:04.441+0000 7f0cefc1c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:04.444 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:04.441+0000 7f0cefc1c780 -1 bdev(0x4136c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:04.445 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:04.441+0000 7f0cefc1c780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T16:24:04.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T16:24:04.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:04.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T16:24:04.757 INFO:tasks.workunit.client.0.trial127.stdout:adding osd6 key to auth repository 2026-02-19T16:24:04.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:05.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T16:24:05.080 INFO:tasks.workunit.client.0.trial127.stdout:start osd.6 2026-02-19T16:24:05.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:05.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:05.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:05.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:05.094 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:05.091+0000 7fa1abe3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:05.095 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:05.093+0000 7fa1abe3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:05.096 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:05.093+0000 7fa1abe3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:05.234 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:05.231+0000 7fa1abe3d780 -1 Falling back to public interface 2026-02-19T16:24:05.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T16:24:05.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:05.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T16:24:05.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:05.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:05.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:05.306 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:05.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:05.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:05.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:24:05.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:05.539 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:05.536+0000 7fa1abe3d780 -1 osd.6 0 log_to_monitors true 2026-02-19T16:24:06.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:06.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:06.534 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:06.534 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:06.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:06.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:24:06.761 INFO:tasks.workunit.client.0.trial127.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3045022368,v1:127.0.0.1:6851/3045022368] [v2:127.0.0.1:6852/3045022368,v1:127.0.0.1:6853/3045022368] exists,up 66274346-d2d5-4ac0-bde0-c4a046d9f44e 2026-02-19T16:24:06.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:06.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:06.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:06.761 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:24:06.761 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:24:06.761 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:24:06.762 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:06.762 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:06.762 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:06.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:24:06.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:24:06.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:24:06.809 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:24:06.812 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:24:06.815 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:23:49.933+0000 7f93e842b780 0 load: jerasure load: lrc 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T16:24:06.817 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:24:06.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T16:24:07.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:24:07.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:24:07.509 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:24:07.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:24:08.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:24:08.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:24:08.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:24:08.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:24:08.520 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:24:08.521 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:24:08.521 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:24:08.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:24:08.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:24:08.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:24:08.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:24:08.608 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:24:08.608 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:24:08.608 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:24:08.608 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:24:08.608 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:24:08.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:24:08.830 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:08.830 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:08.831 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:24:08.831 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:08.831 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:08.831 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:24:08.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:24:08.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:08.831 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:24:08.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T16:24:08.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T16:24:08.897 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T16:24:08.897 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:08.897 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:24:08.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-19T16:24:08.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-19T16:24:08.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558' 2026-02-19T16:24:08.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:08.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:24:09.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T16:24:09.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T16:24:09.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333' 2026-02-19T16:24:09.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:09.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:24:09.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-19T16:24:09.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-19T16:24:09.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701' 2026-02-19T16:24:09.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:09.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:24:09.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-19T16:24:09.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-19T16:24:09.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-146028888068' 2026-02-19T16:24:09.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:09.152 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:24:09.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-19T16:24:09.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-19T16:24:09.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-146028888068 5-184683593732' 2026-02-19T16:24:09.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:09.217 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:24:09.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332099 2026-02-19T16:24:09.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332099 2026-02-19T16:24:09.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-146028888068 5-184683593732 6-219043332099' 2026-02-19T16:24:09.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:09.282 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-19T16:24:09.282 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:09.283 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:24:09.283 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T16:24:09.284 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:09.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-19T16:24:09.285 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T16:24:09.285 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803782 2026-02-19T16:24:09.285 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:09.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-19T16:24:09.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:24:10.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:24:10.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:10.743 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T16:24:10.743 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:10.744 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-19T16:24:10.744 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:10.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:24:10.745 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-19T16:24:10.745 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:10.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-19T16:24:10.747 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-19T16:24:10.747 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607558 2026-02-19T16:24:10.747 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:24:10.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-19T16:24:10.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:10.977 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-19T16:24:10.977 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:10.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:24:10.978 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T16:24:10.978 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:10.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-19T16:24:10.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T16:24:10.980 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411333 2026-02-19T16:24:10.980 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:24:11.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T16:24:11.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:11.208 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149701 2026-02-19T16:24:11.208 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:11.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:24:11.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-19T16:24:11.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:11.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149701 2026-02-19T16:24:11.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-19T16:24:11.211 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149701 2026-02-19T16:24:11.211 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:24:11.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-19T16:24:11.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:11.437 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888068 2026-02-19T16:24:11.438 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:11.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:24:11.439 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888068 2026-02-19T16:24:11.439 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:11.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888068 2026-02-19T16:24:11.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-19T16:24:11.440 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 146028888068 2026-02-19T16:24:11.441 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:24:11.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-19T16:24:11.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:11.668 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593732 2026-02-19T16:24:11.668 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:11.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:24:11.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-19T16:24:11.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:11.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593732 2026-02-19T16:24:11.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-19T16:24:11.671 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 184683593732 2026-02-19T16:24:11.672 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:24:11.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-19T16:24:11.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:11.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332099 2026-02-19T16:24:11.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:11.905 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:24:11.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332099 2026-02-19T16:24:11.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:11.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332099 2026-02-19T16:24:11.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332099' 2026-02-19T16:24:11.908 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 219043332099 2026-02-19T16:24:11.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:24:12.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332100 -lt 219043332099 2026-02-19T16:24:12.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:24:12.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:12.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:12.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:24:12.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:24:12.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:24:12.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:24:12.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:24:12.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:24:12.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:24:12.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:24:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:24:12.681 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:24:12.682 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:12.682 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:13.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:24:13.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:24:13.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:24:13.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-19T16:24:13.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-19T16:24:13.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-19T16:24:13.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:24:13.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T16:24:13.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-19T16:24:13.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T16:24:13.228 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:24:13.228 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:24:13.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:24:13.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:24:13.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:24:13.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:24:13.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:13.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:24:13.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T16:24:13.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-19T16:24:13.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:24:13.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:24:13.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=mdata 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:24:13.992 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:24:13.993 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:24:13.993 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:24:14.223 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=5 2026-02-19T16:24:14.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:24:14.225 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/5/type 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-19T16:24:14.227 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-19T16:24:14.228 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:14.228 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:14.228 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:14.228 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.5.asok 2026-02-19T16:24:14.228 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:24:14.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:24:14.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:24:14.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:24:14.275 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.5 2026-02-19T16:24:14.275 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-19T16:24:14.275 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-19T16:24:14.275 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:14.276 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:14.276 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:14.276 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.5.asok 2026-02-19T16:24:14.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:24:14.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-19T16:24:14.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-19T16:24:14.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:24:14.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:14.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T16:24:14.327 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:24:14.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:14.329 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T16:24:14.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T16:24:14.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:24:14.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:24:14.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:24:14.339 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:14.337+0000 7ff41f609780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:14.341 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:14.339+0000 7ff41f609780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:14.342 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:14.340+0000 7ff41f609780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:14.488 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:14.485+0000 7ff41f609780 -1 Falling back to public interface 2026-02-19T16:24:14.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T16:24:14.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:14.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:24:14.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:14.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:14.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:14.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:14.554 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:14.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:14.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:24:14.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:14.822 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:14.820+0000 7ff41f609780 -1 osd.3 58 log_to_monitors true 2026-02-19T16:24:15.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:15.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:15.779 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:15.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:15.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:15.780 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 62 up_thru 62 down_at 59 last_clean_interval [26,58) [v2:127.0.0.1:6826/27982235,v1:127.0.0.1:6827/27982235] [v2:127.0.0.1:6828/27982235,v1:127.0.0.1:6829/27982235] exists,up b4872867-e058-4969-af84-dc7d9d56ec66 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:24:16.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:24:16.005 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:24:16.005 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:24:16.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:24:16.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:24:16.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:24:16.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:24:16.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:24:16.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:24:16.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:24:16.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:24:16.090 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:24:16.319 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:24:16.319 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:16.319 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:16.319 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:24:16.319 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:16.319 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:16.320 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:24:16.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:24:16.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:16.320 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:24:16.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-19T16:24:16.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-19T16:24:16.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786' 2026-02-19T16:24:16.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:16.383 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:24:16.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-19T16:24:16.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-19T16:24:16.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607561' 2026-02-19T16:24:16.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:16.447 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:24:16.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-19T16:24:16.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-19T16:24:16.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337' 2026-02-19T16:24:16.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:16.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:24:16.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=266287972355 2026-02-19T16:24:16.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 266287972355 2026-02-19T16:24:16.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-266287972355' 2026-02-19T16:24:16.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:16.578 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:24:16.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888072 2026-02-19T16:24:16.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888072 2026-02-19T16:24:16.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-266287972355 4-146028888072' 2026-02-19T16:24:16.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:16.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:24:16.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593735 2026-02-19T16:24:16.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593735 2026-02-19T16:24:16.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-266287972355 4-146028888072 5-184683593735' 2026-02-19T16:24:16.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:16.710 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:24:16.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332103 2026-02-19T16:24:16.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332103 2026-02-19T16:24:16.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-266287972355 4-146028888072 5-184683593735 6-219043332103' 2026-02-19T16:24:16.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:16.773 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803786 2026-02-19T16:24:16.774 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:16.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:24:16.775 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-19T16:24:16.775 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:16.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803786 2026-02-19T16:24:16.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-19T16:24:16.777 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803786 2026-02-19T16:24:16.777 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:17.008 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803786 2026-02-19T16:24:17.008 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:24:18.008 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:24:18.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:18.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-19T16:24:18.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:18.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607561 2026-02-19T16:24:18.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:18.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:24:18.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-19T16:24:18.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:18.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607561 2026-02-19T16:24:18.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-19T16:24:18.238 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607561 2026-02-19T16:24:18.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:24:18.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-19T16:24:18.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:18.467 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411337 2026-02-19T16:24:18.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:18.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:24:18.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411337 2026-02-19T16:24:18.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:18.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-19T16:24:18.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-19T16:24:18.470 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411337 2026-02-19T16:24:18.471 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:24:18.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-19T16:24:18.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:18.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-266287972355 2026-02-19T16:24:18.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:18.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:24:18.704 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-266287972355 2026-02-19T16:24:18.704 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:18.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=266287972355 2026-02-19T16:24:18.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 266287972355' 2026-02-19T16:24:18.705 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 266287972355 2026-02-19T16:24:18.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:24:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 266287972355 -lt 266287972355 2026-02-19T16:24:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:18.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888072 2026-02-19T16:24:18.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:18.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:24:18.944 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888072 2026-02-19T16:24:18.944 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:18.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888072 2026-02-19T16:24:18.946 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 146028888072 2026-02-19T16:24:18.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888072' 2026-02-19T16:24:18.947 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:24:19.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888072 -lt 146028888072 2026-02-19T16:24:19.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:19.184 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593735 2026-02-19T16:24:19.184 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:19.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:24:19.185 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593735 2026-02-19T16:24:19.185 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:19.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593735 2026-02-19T16:24:19.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-19T16:24:19.186 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 184683593735 2026-02-19T16:24:19.186 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:24:19.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-19T16:24:19.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:19.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332103 2026-02-19T16:24:19.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:19.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:24:19.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332103 2026-02-19T16:24:19.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:19.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332103 2026-02-19T16:24:19.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332103' 2026-02-19T16:24:19.420 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 219043332103 2026-02-19T16:24:19.420 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:24:19.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332103 -lt 219043332103 2026-02-19T16:24:19.651 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:24:19.652 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:19.652 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:19.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:24:19.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:24:19.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:24:19.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:24:19.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:24:19.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:24:19.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:24:19.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:24:20.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:24:20.196 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:24:20.196 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:20.196 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:24:20.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-19T16:24:20.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:24:20.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:24:20.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:24:20.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:24:20.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:24:20.802 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:24:20.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:24:21.112 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:24:21.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:24:21.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:24:21.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:24:21.231 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:24:21.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:24:21.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:24:21.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:24:21.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:24:21.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:24:21.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:21.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:24:21.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:24:21.236 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:21.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:24:21.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:24:21.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:24:21.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:24:21.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:21.254 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:21.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:24:21.255 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:24:21.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:24:21.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:24:21.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:24:21.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:24:21.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:24:21.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:24:21.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:24:21.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:24:21.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:24:21.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:24:21.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:21.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:24:21.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:24:21.260 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:21.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:24:21.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:24:21.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:24:21.263 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:24:21.263 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:21.263 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:21.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:24:21.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:24:21.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:24:21.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:24:21.265 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:24:21.265 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:21.265 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:21.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:24:21.266 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:24:21.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:24:21.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:24:21.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:24:21.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:24:21.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:21.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:21.288 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:21.288 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:21.288 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:21.288 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:21.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:24:21.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:24:21.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:24:21.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:24:21.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:24:21.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:24:21.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:24:21.312 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:24:21.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:24:21.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:24:21.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:24:21.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:24:21.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:24:21.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:24:21.361 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:24:21.361 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:24:21.361 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:24:21.361 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:21.361 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:21.361 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:21.361 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:24:21.362 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:24:21.362 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:24:21.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:24:21.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:24:21.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:24:21.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:24:21.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:24:21.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:24:21.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:24:21.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:24:21.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:21.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:21.506 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:21.507 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:21.507 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:21.507 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:21.507 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:24:21.508 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:24:21.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:24:21.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:24:21.617 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:24:21.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:24:22.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:24:22.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:24:22.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:24:22.627 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:22.627 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:22.627 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:22.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:24:22.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:24:22.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:24:22.666 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:24:22.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:24:22.671 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:24:21.302+0000 7fcf3722ec00 0 load: jerasure load: lrc 2026-02-19T16:24:22.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-19T16:24:22.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-19T16:24:22.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-19T16:24:22.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-19T16:24:22.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-19T16:24:22.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:24:22.673 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-19T16:24:22.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-19T16:24:22.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:22.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:24:22.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:22.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:22.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:24:22.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:22.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:22.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:22.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:22.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:22.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:22.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:22.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:22.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:24:22.680 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:22.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=825f9f95-8ebc-48c8-b864-7b7dd590d12a 2026-02-19T16:24:22.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 825f9f95-8ebc-48c8-b864-7b7dd590d12a' 2026-02-19T16:24:22.681 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 825f9f95-8ebc-48c8-b864-7b7dd590d12a 2026-02-19T16:24:22.681 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:22.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA2OZdpS3blKBAAgQnPongZe+ye85Nui6wStA== 2026-02-19T16:24:22.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA2OZdpS3blKBAAgQnPongZe+ye85Nui6wStA=="}' 2026-02-19T16:24:22.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 825f9f95-8ebc-48c8-b864-7b7dd590d12a -i td/test-erasure-eio/0/new.json 2026-02-19T16:24:22.791 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:22.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:24:22.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA2OZdpS3blKBAAgQnPongZe+ye85Nui6wStA== --osd-uuid 825f9f95-8ebc-48c8-b864-7b7dd590d12a 2026-02-19T16:24:22.815 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:22.814+0000 7fd859f3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:22.817 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:22.815+0000 7fd859f3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:22.818 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:22.816+0000 7fd859f3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:22.818 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:22.816+0000 7fd859f3d780 -1 bdev(0x43fcc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:22.819 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:22.816+0000 7fd859f3d780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:24:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:24:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:24:23.235 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:24:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:23.339 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:24:23.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:24:23.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:23.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:23.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:23.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:23.354 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:23.352+0000 7f88f0e66780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:23.355 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:23.353+0000 7f88f0e66780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:23.356 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:23.354+0000 7f88f0e66780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:23.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:24:23.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:23.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:24:23.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:23.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:23.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:23.439 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:23.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:23.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:23.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:24:23.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:23.573 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:23.571+0000 7f88f0e66780 -1 Falling back to public interface 2026-02-19T16:24:23.832 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:23.830+0000 7f88f0e66780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:24:24.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:24.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:24.536 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:24.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:24.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:24.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:24:24.767 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3311671133,v1:127.0.0.1:6803/3311671133] [v2:127.0.0.1:6804/3311671133,v1:127.0.0.1:6805/3311671133] exists,up 825f9f95-8ebc-48c8-b864-7b7dd590d12a 2026-02-19T16:24:24.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:24.768 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:24.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:24:24.770 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:24.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bed605aa-0811-44e0-a210-f1710cbf0a61 2026-02-19T16:24:24.770 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 bed605aa-0811-44e0-a210-f1710cbf0a61 2026-02-19T16:24:24.771 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 bed605aa-0811-44e0-a210-f1710cbf0a61' 2026-02-19T16:24:24.771 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:24.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA4OZdpS8OBLhAAbVOWYhtv02YVKWIwnmGfTA== 2026-02-19T16:24:24.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA4OZdpS8OBLhAAbVOWYhtv02YVKWIwnmGfTA=="}' 2026-02-19T16:24:24.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bed605aa-0811-44e0-a210-f1710cbf0a61 -i td/test-erasure-eio/1/new.json 2026-02-19T16:24:25.016 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:25.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:24:25.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA4OZdpS8OBLhAAbVOWYhtv02YVKWIwnmGfTA== --osd-uuid bed605aa-0811-44e0-a210-f1710cbf0a61 2026-02-19T16:24:25.041 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.039+0000 7fac10838780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:25.043 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.041+0000 7fac10838780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:25.044 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.042+0000 7fac10838780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:25.044 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.042+0000 7fac10838780 -1 bdev(0x28ccc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:25.045 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.043+0000 7fac10838780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:24:25.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:24:25.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:25.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:24:25.396 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:24:25.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:25.719 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:24:25.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:24:25.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:25.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:25.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:25.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:25.735 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.733+0000 7f5584fe1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:25.736 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.734+0000 7f5584fe1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:25.737 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.735+0000 7f5584fe1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:25.883 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:25.881+0000 7f5584fe1780 -1 Falling back to public interface 2026-02-19T16:24:25.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:24:25.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:25.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:24:25.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:25.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:25.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:25.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:25.947 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:25.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:25.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:24:26.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:26.222 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:26.220+0000 7f5584fe1780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:24:27.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:27.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:27.175 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:27.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:27.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:27.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:24:27.401 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/217791831,v1:127.0.0.1:6811/217791831] [v2:127.0.0.1:6812/217791831,v1:127.0.0.1:6813/217791831] exists,up bed605aa-0811-44e0-a210-f1710cbf0a61 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:27.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:27.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:24:27.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:27.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5c89a720-549b-4573-87c5-5220b6b4f200 2026-02-19T16:24:27.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5c89a720-549b-4573-87c5-5220b6b4f200' 2026-02-19T16:24:27.405 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 5c89a720-549b-4573-87c5-5220b6b4f200 2026-02-19T16:24:27.405 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:27.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA7OZdpZLunGBAAnGDDY6za/d/EUpvgy4sQsg== 2026-02-19T16:24:27.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA7OZdpZLunGBAAnGDDY6za/d/EUpvgy4sQsg=="}' 2026-02-19T16:24:27.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5c89a720-549b-4573-87c5-5220b6b4f200 -i td/test-erasure-eio/2/new.json 2026-02-19T16:24:27.646 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:24:27.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:24:27.655 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA7OZdpZLunGBAAnGDDY6za/d/EUpvgy4sQsg== --osd-uuid 5c89a720-549b-4573-87c5-5220b6b4f200 2026-02-19T16:24:27.671 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:27.669+0000 7fe98802f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:27.672 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:27.670+0000 7fe98802f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:27.673 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:27.671+0000 7fe98802f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:27.674 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:27.672+0000 7fe98802f780 -1 bdev(0x33d8c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:27.674 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:27.672+0000 7fe98802f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:24:27.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:24:27.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:27.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:24:27.988 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:24:27.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:28.306 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:24:28.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:24:28.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:28.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:28.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:28.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:28.321 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:28.320+0000 7feb2b1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:28.323 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:28.321+0000 7feb2b1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:28.324 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:28.322+0000 7feb2b1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:28.451 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:28.449+0000 7feb2b1d8780 -1 Falling back to public interface 2026-02-19T16:24:28.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:24:28.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:28.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:24:28.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:28.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:28.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:28.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:28.539 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:28.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:28.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:24:28.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:28.800 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:28.798+0000 7feb2b1d8780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:24:29.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:29.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:29.768 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:29.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:29.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:29.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:24:29.995 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2748817298,v1:127.0.0.1:6819/2748817298] [v2:127.0.0.1:6820/2748817298,v1:127.0.0.1:6821/2748817298] exists,up 5c89a720-549b-4573-87c5-5220b6b4f200 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:29.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:30.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:30.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e9ac5027-c004-4613-a693-0070e58af7ed 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e9ac5027-c004-4613-a693-0070e58af7ed' 2026-02-19T16:24:30.002 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 e9ac5027-c004-4613-a693-0070e58af7ed 2026-02-19T16:24:30.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:30.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+OZdpgSJsABAAd9cSlqnTlQm2fp/UztW9Jg== 2026-02-19T16:24:30.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+OZdpgSJsABAAd9cSlqnTlQm2fp/UztW9Jg=="}' 2026-02-19T16:24:30.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e9ac5027-c004-4613-a693-0070e58af7ed -i td/test-erasure-eio/3/new.json 2026-02-19T16:24:30.244 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:24:30.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:24:30.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA+OZdpgSJsABAAd9cSlqnTlQm2fp/UztW9Jg== --osd-uuid e9ac5027-c004-4613-a693-0070e58af7ed 2026-02-19T16:24:30.268 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.267+0000 7f3a1aa0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:30.270 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.268+0000 7f3a1aa0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:30.271 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.269+0000 7f3a1aa0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:30.271 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.269+0000 7f3a1aa0e780 -1 bdev(0x39b4c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:30.271 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.269+0000 7f3a1aa0e780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:24:30.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:24:30.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:30.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:24:30.584 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:24:30.585 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:30.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:24:30.907 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:24:30.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:30.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:30.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:30.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:30.921 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.920+0000 7fdc4f625780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:30.923 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.921+0000 7fdc4f625780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:30.923 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:30.922+0000 7fdc4f625780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:31.106 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:31.103+0000 7fdc4f625780 -1 Falling back to public interface 2026-02-19T16:24:31.134 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:24:31.134 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:31.134 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:24:31.134 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:31.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:31.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:31.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:31.135 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:31.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:31.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:24:31.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:31.402 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:31.400+0000 7fdc4f625780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:24:32.368 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:32.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:32.369 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:32.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:32.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:32.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:24:32.599 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/729026997,v1:127.0.0.1:6827/729026997] [v2:127.0.0.1:6828/729026997,v1:127.0.0.1:6829/729026997] exists,up e9ac5027-c004-4613-a693-0070e58af7ed 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:32.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:32.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T16:24:32.602 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:32.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cb9a6b6e-2d0b-4b74-8734-188f5a3c3d1d 2026-02-19T16:24:32.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 cb9a6b6e-2d0b-4b74-8734-188f5a3c3d1d' 2026-02-19T16:24:32.603 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 cb9a6b6e-2d0b-4b74-8734-188f5a3c3d1d 2026-02-19T16:24:32.603 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:32.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBAOZdpPvV0JBAAa7ldlh0tNwqg21dPZGQI1A== 2026-02-19T16:24:32.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBAOZdpPvV0JBAAa7ldlh0tNwqg21dPZGQI1A=="}' 2026-02-19T16:24:32.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cb9a6b6e-2d0b-4b74-8734-188f5a3c3d1d -i td/test-erasure-eio/4/new.json 2026-02-19T16:24:32.846 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:24:32.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T16:24:32.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAOZdpPvV0JBAAa7ldlh0tNwqg21dPZGQI1A== --osd-uuid cb9a6b6e-2d0b-4b74-8734-188f5a3c3d1d 2026-02-19T16:24:32.871 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:32.869+0000 7fe522ddf780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:32.872 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:32.871+0000 7fe522ddf780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:32.873 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:32.872+0000 7fe522ddf780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:32.874 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:32.872+0000 7fe522ddf780 -1 bdev(0x3ed4c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:32.874 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:32.872+0000 7fe522ddf780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T16:24:33.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T16:24:33.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:33.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:24:33.193 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:24:33.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:33.531 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:24:33.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:24:33.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:33.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:33.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:33.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:33.546 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:33.544+0000 7fe83f645780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:33.547 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:33.545+0000 7fe83f645780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:33.548 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:33.546+0000 7fe83f645780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:33.722 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:33.720+0000 7fe83f645780 -1 Falling back to public interface 2026-02-19T16:24:33.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:24:33.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:33.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:24:33.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:33.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:33.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:33.758 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:33.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:33.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:33.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:24:33.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:34.082 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:34.080+0000 7fe83f645780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:24:34.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:34.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:34.986 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:34.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:34.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:34.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:24:35.213 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1312488518,v1:127.0.0.1:6835/1312488518] [v2:127.0.0.1:6836/1312488518,v1:127.0.0.1:6837/1312488518] exists,up cb9a6b6e-2d0b-4b74-8734-188f5a3c3d1d 2026-02-19T16:24:35.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:35.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:35.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:35.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:35.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T16:24:35.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:35.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0fa130ad-6046-48f5-9ae3-97b6e475fea5 2026-02-19T16:24:35.221 INFO:tasks.workunit.client.0.trial127.stdout:add osd5 0fa130ad-6046-48f5-9ae3-97b6e475fea5 2026-02-19T16:24:35.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 0fa130ad-6046-48f5-9ae3-97b6e475fea5' 2026-02-19T16:24:35.221 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:35.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDOZdp9yBvDRAAbZV6agXLZ++d1mc0PYwePQ== 2026-02-19T16:24:35.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDOZdp9yBvDRAAbZV6agXLZ++d1mc0PYwePQ=="}' 2026-02-19T16:24:35.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0fa130ad-6046-48f5-9ae3-97b6e475fea5 -i td/test-erasure-eio/5/new.json 2026-02-19T16:24:35.464 INFO:tasks.workunit.client.0.trial127.stdout:5 2026-02-19T16:24:35.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T16:24:35.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDOZdp9yBvDRAAbZV6agXLZ++d1mc0PYwePQ== --osd-uuid 0fa130ad-6046-48f5-9ae3-97b6e475fea5 2026-02-19T16:24:35.489 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:35.488+0000 7ff35a44e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:35.490 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:35.489+0000 7ff35a44e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:35.491 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:35.490+0000 7ff35a44e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:35.492 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:35.490+0000 7ff35a44e780 -1 bdev(0x3fa0c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:35.492 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:35.490+0000 7ff35a44e780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T16:24:35.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T16:24:35.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:35.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T16:24:35.839 INFO:tasks.workunit.client.0.trial127.stdout:adding osd5 key to auth repository 2026-02-19T16:24:35.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:36.163 INFO:tasks.workunit.client.0.trial127.stdout:start osd.5 2026-02-19T16:24:36.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T16:24:36.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:36.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:36.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:36.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:36.178 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:36.176+0000 7fe3d7e5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:36.180 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:36.177+0000 7fe3d7e5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:36.181 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:36.178+0000 7fe3d7e5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:36.322 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:36.319+0000 7fe3d7e5a780 -1 Falling back to public interface 2026-02-19T16:24:36.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T16:24:36.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:36.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T16:24:36.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:36.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:36.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:36.392 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:36.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:36.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:36.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:24:36.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:36.653 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:36.650+0000 7fe3d7e5a780 -1 osd.5 0 log_to_monitors true 2026-02-19T16:24:37.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:37.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:37.621 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:37.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:37.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:37.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:24:37.848 INFO:tasks.workunit.client.0.trial127.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2811258621,v1:127.0.0.1:6843/2811258621] [v2:127.0.0.1:6844/2811258621,v1:127.0.0.1:6845/2811258621] exists,up 0fa130ad-6046-48f5-9ae3-97b6e475fea5 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:37.849 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:37.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T16:24:37.851 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:37.852 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7a924989-0316-4a7b-8338-0218519406dc 2026-02-19T16:24:37.852 INFO:tasks.workunit.client.0.trial127.stdout:add osd6 7a924989-0316-4a7b-8338-0218519406dc 2026-02-19T16:24:37.852 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 7a924989-0316-4a7b-8338-0218519406dc' 2026-02-19T16:24:37.852 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:37.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBFOZdplfROMxAAvJrcto1fLDEJJtNSxiTefA== 2026-02-19T16:24:37.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBFOZdplfROMxAAvJrcto1fLDEJJtNSxiTefA=="}' 2026-02-19T16:24:37.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7a924989-0316-4a7b-8338-0218519406dc -i td/test-erasure-eio/6/new.json 2026-02-19T16:24:38.097 INFO:tasks.workunit.client.0.trial127.stdout:6 2026-02-19T16:24:38.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T16:24:38.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBFOZdplfROMxAAvJrcto1fLDEJJtNSxiTefA== --osd-uuid 7a924989-0316-4a7b-8338-0218519406dc 2026-02-19T16:24:38.122 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.119+0000 7f1e849fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:38.123 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.121+0000 7f1e849fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:38.124 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.122+0000 7f1e849fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:38.124 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.122+0000 7f1e849fb780 -1 bdev(0x373ec00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:38.125 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.122+0000 7f1e849fb780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T16:24:38.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T16:24:38.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:38.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T16:24:38.450 INFO:tasks.workunit.client.0.trial127.stdout:adding osd6 key to auth repository 2026-02-19T16:24:38.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:38.768 INFO:tasks.workunit.client.0.trial127.stdout:start osd.6 2026-02-19T16:24:38.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T16:24:38.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:38.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:38.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:38.770 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:38.783 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.780+0000 7f66fc638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:38.784 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.781+0000 7f66fc638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:38.785 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.782+0000 7f66fc638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:38.927 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:38.924+0000 7f66fc638780 -1 Falling back to public interface 2026-02-19T16:24:38.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T16:24:38.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:38.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T16:24:38.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:38.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:38.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:38.993 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:38.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:38.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:38.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:24:39.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:39.260 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:39.257+0000 7f66fc638780 -1 osd.6 0 log_to_monitors true 2026-02-19T16:24:40.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:40.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:40.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:40.224 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:40.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:40.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:24:40.450 INFO:tasks.workunit.client.0.trial127.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/667290119,v1:127.0.0.1:6851/667290119] [v2:127.0.0.1:6852/667290119,v1:127.0.0.1:6853/667290119] exists,up 7a924989-0316-4a7b-8338-0218519406dc 2026-02-19T16:24:40.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:40.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:40.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:40.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:40.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-19T16:24:40.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:40.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:40.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:40.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=78b0a256-c264-4d2b-a49a-6411cbbceaac 2026-02-19T16:24:40.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 78b0a256-c264-4d2b-a49a-6411cbbceaac' 2026-02-19T16:24:40.458 INFO:tasks.workunit.client.0.trial127.stdout:add osd7 78b0a256-c264-4d2b-a49a-6411cbbceaac 2026-02-19T16:24:40.458 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:40.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIOZdpndCIGxAAf+Wklnhi819PLJLAJr1oKw== 2026-02-19T16:24:40.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIOZdpndCIGxAAf+Wklnhi819PLJLAJr1oKw=="}' 2026-02-19T16:24:40.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 78b0a256-c264-4d2b-a49a-6411cbbceaac -i td/test-erasure-eio/7/new.json 2026-02-19T16:24:40.691 INFO:tasks.workunit.client.0.trial127.stdout:7 2026-02-19T16:24:40.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-19T16:24:40.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIOZdpndCIGxAAf+Wklnhi819PLJLAJr1oKw== --osd-uuid 78b0a256-c264-4d2b-a49a-6411cbbceaac 2026-02-19T16:24:40.716 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:40.714+0000 7ff08ca25780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:40.718 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:40.715+0000 7ff08ca25780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:40.719 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:40.716+0000 7ff08ca25780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:40.720 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:40.716+0000 7ff08ca25780 -1 bdev(0x2c6cc00 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:40.720 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:40.716+0000 7ff08ca25780 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-19T16:24:41.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-19T16:24:41.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:41.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-19T16:24:41.061 INFO:tasks.workunit.client.0.trial127.stdout:adding osd7 key to auth repository 2026-02-19T16:24:41.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:41.381 INFO:tasks.workunit.client.0.trial127.stdout:start osd.7 2026-02-19T16:24:41.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-19T16:24:41.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:41.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:41.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:41.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:41.397 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:41.394+0000 7f8bfb7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:41.398 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:41.395+0000 7f8bfb7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:41.399 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:41.396+0000 7f8bfb7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:41.538 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:41.535+0000 7f8bfb7d8780 -1 Falling back to public interface 2026-02-19T16:24:41.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-19T16:24:41.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:41.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-19T16:24:41.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:41.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:41.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:41.613 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:41.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:41.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:41.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T16:24:41.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:41.872 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:41.869+0000 7f8bfb7d8780 -1 osd.7 0 log_to_monitors true 2026-02-19T16:24:42.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:42.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:42.840 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:42.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:42.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:42.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T16:24:43.068 INFO:tasks.workunit.client.0.trial127.stdout:osd.7 up in weight 1 up_from 58 up_thru 59 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/180389249,v1:127.0.0.1:6859/180389249] [v2:127.0.0.1:6860/180389249,v1:127.0.0.1:6861/180389249] exists,up 78b0a256-c264-4d2b-a49a-6411cbbceaac 2026-02-19T16:24:43.071 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:24:43.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:43.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:24:43.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:24:43.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:24:43.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:24:43.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:24:43.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-19T16:24:43.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:24:43.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a0502fcc-5ef7-4470-bfa6-0aea4bf4aaa2 2026-02-19T16:24:43.075 INFO:tasks.workunit.client.0.trial127.stdout:add osd8 a0502fcc-5ef7-4470-bfa6-0aea4bf4aaa2 2026-02-19T16:24:43.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 a0502fcc-5ef7-4470-bfa6-0aea4bf4aaa2' 2026-02-19T16:24:43.076 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:24:43.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBLOZdp3djFBBAA3bIYE3M0X5RcgfJUNN9/jw== 2026-02-19T16:24:43.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBLOZdp3djFBBAA3bIYE3M0X5RcgfJUNN9/jw=="}' 2026-02-19T16:24:43.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a0502fcc-5ef7-4470-bfa6-0aea4bf4aaa2 -i td/test-erasure-eio/8/new.json 2026-02-19T16:24:43.312 INFO:tasks.workunit.client.0.trial127.stdout:8 2026-02-19T16:24:43.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-19T16:24:43.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLOZdp3djFBBAA3bIYE3M0X5RcgfJUNN9/jw== --osd-uuid a0502fcc-5ef7-4470-bfa6-0aea4bf4aaa2 2026-02-19T16:24:43.337 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:43.334+0000 7fc618662780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:43.338 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:43.336+0000 7fc618662780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:43.339 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:43.337+0000 7fc618662780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:43.340 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:43.337+0000 7fc618662780 -1 bdev(0x2b5ac00 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-19T16:24:43.340 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:43.337+0000 7fc618662780 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-19T16:24:43.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-19T16:24:43.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:24:43.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-19T16:24:43.677 INFO:tasks.workunit.client.0.trial127.stdout:adding osd8 key to auth repository 2026-02-19T16:24:43.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:24:43.993 INFO:tasks.workunit.client.0.trial127.stdout:start osd.8 2026-02-19T16:24:43.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-19T16:24:43.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:24:43.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:24:43.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:24:43.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:24:44.008 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:44.006+0000 7f1be5e39780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:44.010 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:44.007+0000 7f1be5e39780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:44.011 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:44.008+0000 7f1be5e39780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:24:44.204 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:44.201+0000 7f1be5e39780 -1 Falling back to public interface 2026-02-19T16:24:44.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-19T16:24:44.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:24:44.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-19T16:24:44.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:24:44.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:24:44.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:44.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:24:44.223 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:24:44.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:44.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T16:24:44.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:24:44.552 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:24:44.549+0000 7f1be5e39780 -1 osd.8 0 log_to_monitors true 2026-02-19T16:24:45.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:24:45.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:24:45.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:24:45.449 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:24:45.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:24:45.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T16:24:45.677 INFO:tasks.workunit.client.0.trial127.stdout:osd.8 up in weight 1 up_from 66 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/3545289477,v1:127.0.0.1:6867/3545289477] [v2:127.0.0.1:6868/3545289477,v1:127.0.0.1:6869/3545289477] exists,up a0502fcc-5ef7-4470-bfa6-0aea4bf4aaa2 2026-02-19T16:24:45.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:24:45.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:24:45.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:24:45.679 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:24:45.679 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:24:45.679 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:24:45.680 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:45.680 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:45.680 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:45.680 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:24:45.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:24:45.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:24:45.725 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:24:45.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:24:45.732 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:24:23.573+0000 7f88f0e66780 0 load: jerasure load: lrc 2026-02-19T16:24:45.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-19T16:24:45.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-19T16:24:45.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:24:45.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:24:45.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-19T16:24:45.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:24:45.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-19T16:24:45.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:24:45.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-19T16:24:46.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:24:46.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:24:46.400 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:24:46.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:24:47.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:24:47.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:24:47.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:24:47.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:24:47.410 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:24:47.411 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:24:47.411 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:24:47.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:24:47.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:24:47.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:24:47.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:24:47.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:24:47.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:24:47.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:24:47.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:24:47.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:24:47.741 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:24:47.741 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:47.741 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:47.741 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr:8' 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:47.742 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:24:47.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T16:24:47.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T16:24:47.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T16:24:47.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:47.808 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:24:47.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-19T16:24:47.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-19T16:24:47.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855' 2026-02-19T16:24:47.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:47.871 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:24:47.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378630 2026-02-19T16:24:47.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378630 2026-02-19T16:24:47.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378630' 2026-02-19T16:24:47.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:47.939 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:24:48.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T16:24:48.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T16:24:48.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378630 3-111669149702' 2026-02-19T16:24:48.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:48.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:24:48.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888069 2026-02-19T16:24:48.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888069 2026-02-19T16:24:48.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378630 3-111669149702 4-146028888069' 2026-02-19T16:24:48.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:48.065 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:24:48.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626437 2026-02-19T16:24:48.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626437 2026-02-19T16:24:48.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378630 3-111669149702 4-146028888069 5-180388626437' 2026-02-19T16:24:48.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:48.131 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:24:48.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364804 2026-02-19T16:24:48.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364804 2026-02-19T16:24:48.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378630 3-111669149702 4-146028888069 5-180388626437 6-214748364804' 2026-02-19T16:24:48.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:48.195 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:24:48.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103172 2026-02-19T16:24:48.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103172 2026-02-19T16:24:48.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378630 3-111669149702 4-146028888069 5-180388626437 6-214748364804 7-249108103172' 2026-02-19T16:24:48.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:48.260 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:24:48.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=283467841539 2026-02-19T16:24:48.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 283467841539 2026-02-19T16:24:48.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378630 3-111669149702 4-146028888069 5-180388626437 6-214748364804 7-249108103172 8-283467841539' 2026-02-19T16:24:48.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:48.328 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-19T16:24:48.328 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:48.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:24:48.329 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T16:24:48.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:48.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-19T16:24:48.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T16:24:48.331 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803784 2026-02-19T16:24:48.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:48.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-19T16:24:48.559 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:24:49.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:24:49.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:49.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T16:24:49.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:49.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574855 2026-02-19T16:24:49.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:49.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:24:49.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-19T16:24:49.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:49.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-19T16:24:49.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-19T16:24:49.792 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 55834574855 2026-02-19T16:24:49.793 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:24:50.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-19T16:24:50.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:50.022 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378630 2026-02-19T16:24:50.022 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:50.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:24:50.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378630 2026-02-19T16:24:50.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:50.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378630 2026-02-19T16:24:50.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378630' 2026-02-19T16:24:50.025 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 81604378630 2026-02-19T16:24:50.026 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:24:50.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378630 -lt 81604378630 2026-02-19T16:24:50.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:50.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-19T16:24:50.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:50.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:24:50.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T16:24:50.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:50.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-19T16:24:50.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T16:24:50.259 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149702 2026-02-19T16:24:50.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:24:50.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-19T16:24:50.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:50.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888069 2026-02-19T16:24:50.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:50.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:24:50.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888069 2026-02-19T16:24:50.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:50.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888069 2026-02-19T16:24:50.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888069' 2026-02-19T16:24:50.487 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 146028888069 2026-02-19T16:24:50.487 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:24:50.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888069 -lt 146028888069 2026-02-19T16:24:50.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:50.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626437 2026-02-19T16:24:50.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:50.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:24:50.717 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626437 2026-02-19T16:24:50.717 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:50.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626437 2026-02-19T16:24:50.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626437' 2026-02-19T16:24:50.719 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 180388626437 2026-02-19T16:24:50.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:24:50.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626437 -lt 180388626437 2026-02-19T16:24:50.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:50.950 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364804 2026-02-19T16:24:50.951 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:50.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:24:50.952 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364804 2026-02-19T16:24:50.952 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:50.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364804 2026-02-19T16:24:50.953 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 214748364804 2026-02-19T16:24:50.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364804' 2026-02-19T16:24:50.954 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:24:51.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364804 -lt 214748364804 2026-02-19T16:24:51.180 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:51.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-249108103172 2026-02-19T16:24:51.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:51.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-19T16:24:51.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-249108103172 2026-02-19T16:24:51.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:51.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=249108103172 2026-02-19T16:24:51.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 249108103172' 2026-02-19T16:24:51.183 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.7 seq 249108103172 2026-02-19T16:24:51.183 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T16:24:51.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 249108103172 -lt 249108103172 2026-02-19T16:24:51.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:51.413 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-283467841539 2026-02-19T16:24:51.413 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:51.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:24:51.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-283467841539 2026-02-19T16:24:51.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:51.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=283467841539 2026-02-19T16:24:51.415 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.8 seq 283467841539 2026-02-19T16:24:51.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 283467841539' 2026-02-19T16:24:51.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:24:51.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 283467841540 -lt 283467841539 2026-02-19T16:24:51.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:24:51.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:51.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:51.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:24:51.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:24:51.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:24:51.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:24:51.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:24:51.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:24:51.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:24:51.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:24:52.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:24:52.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:24:52.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:52.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:24:52.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:24:52.517 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:24:52.518 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:24:52.518 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:7' 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:24:52.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-19T16:24:52.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:24:52.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T16:24:52.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:24:52.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T16:24:52.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:24:52.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T16:24:52.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:24:52.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:24:52.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:24:52.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T16:24:52.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:24:52.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:24:52.753 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:52.753 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:52.753 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:52.753 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:24:52.753 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:24:52.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:24:52.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:24:52.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:24:52.802 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:24:52.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-19T16:24:52.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-19T16:24:52.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:24:52.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=3 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:24:52.854 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:24:52.855 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:24:52.855 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:24:53.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:24:53.087 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:53.087 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:53.087 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:24:53.087 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr:7' 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=0 2026-02-19T16:24:53.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:24:53.089 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/0/type 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T16:24:53.090 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:24:53.091 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:24:53.091 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:53.092 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:53.092 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:53.092 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:24:53.092 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:24:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:24:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:24:53.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:24:53.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.0 2026-02-19T16:24:53.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:24:53.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:24:53.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:53.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:53.141 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:53.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:24:53.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:24:53.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-19T16:24:53.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-19T16:24:53.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:24:53.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:24:53.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=4 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-19T16:24:53.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:24:53.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:24:53.190 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:24:53.190 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:24:53.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:7' 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T16:24:53.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T16:24:53.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:24:53.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=6 2026-02-19T16:24:53.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:24:53.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/6/type 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-19T16:24:53.419 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-19T16:24:53.420 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:53.420 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:53.420 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:53.420 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.6.asok 2026-02-19T16:24:53.420 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:24:53.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:24:53.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:24:53.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:24:53.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.6 2026-02-19T16:24:53.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-19T16:24:53.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-19T16:24:53.470 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:24:53.470 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:53.470 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:53.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.6.asok 2026-02-19T16:24:53.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:24:53.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-19T16:24:53.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-19T16:24:53.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:24:53.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:24:53.516 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:24:53.516 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:53.744 INFO:tasks.workunit.client.0.trial127.stderr:7' 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:24:53.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:24:53.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:24:53.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-19T16:24:54.081 INFO:tasks.workunit.client.0.trial127.stderr:osd.7 is already down. 2026-02-19T16:24:54.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-19T16:24:54.371 INFO:tasks.workunit.client.0.trial127.stderr:osd.7 is already out. 2026-02-19T16:24:54.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-19T16:24:54.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:24:54.380 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:24:54.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:24:54.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:24:54.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:24:54.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:24:54.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:24:54.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:24:54.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:24:54.692 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:24:54.692 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:24:54.692 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr:6 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr:7 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr:8' 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:54.693 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:24:54.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-19T16:24:54.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-19T16:24:54.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-19T16:24:54.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:54.761 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:24:54.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-19T16:24:54.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-19T16:24:54.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-55834574858' 2026-02-19T16:24:54.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:54.826 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:24:54.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378633 2026-02-19T16:24:54.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378633 2026-02-19T16:24:54.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-81604378633' 2026-02-19T16:24:54.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:54.890 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:24:54.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149705 2026-02-19T16:24:54.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149705 2026-02-19T16:24:54.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-81604378633 3-111669149705' 2026-02-19T16:24:54.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:54.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:24:55.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888072 2026-02-19T16:24:55.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888072 2026-02-19T16:24:55.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-81604378633 3-111669149705 4-146028888072' 2026-02-19T16:24:55.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:55.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:24:55.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626440 2026-02-19T16:24:55.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626440 2026-02-19T16:24:55.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-81604378633 3-111669149705 4-146028888072 5-180388626440' 2026-02-19T16:24:55.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:55.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:24:55.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364807 2026-02-19T16:24:55.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364807 2026-02-19T16:24:55.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-81604378633 3-111669149705 4-146028888072 5-180388626440 6-214748364807' 2026-02-19T16:24:55.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:55.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-19T16:24:55.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-19T16:24:55.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-19T16:24:55.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-19T16:24:55.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-19T16:24:55.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:24:55.199 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-19T16:24:55.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=283467841542 2026-02-19T16:24:55.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 283467841542 2026-02-19T16:24:55.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-81604378633 3-111669149705 4-146028888072 5-180388626440 6-214748364807 8-283467841542' 2026-02-19T16:24:55.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:55.267 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-19T16:24:55.268 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:55.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:24:55.269 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-19T16:24:55.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:55.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-19T16:24:55.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-19T16:24:55.271 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803787 2026-02-19T16:24:55.271 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:55.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-19T16:24:55.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:24:56.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:24:56.498 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:24:56.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803787 2026-02-19T16:24:56.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:56.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574858 2026-02-19T16:24:56.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:56.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:24:56.728 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574858 2026-02-19T16:24:56.728 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:56.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574858 2026-02-19T16:24:56.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-19T16:24:56.730 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 55834574858 2026-02-19T16:24:56.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:24:56.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-19T16:24:56.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:56.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378633 2026-02-19T16:24:56.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:56.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:24:56.960 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378633 2026-02-19T16:24:56.960 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:56.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378633 2026-02-19T16:24:56.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378633' 2026-02-19T16:24:56.961 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 81604378633 2026-02-19T16:24:56.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:24:57.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378634 -lt 81604378633 2026-02-19T16:24:57.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:57.188 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149705 2026-02-19T16:24:57.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:57.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:24:57.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149705 2026-02-19T16:24:57.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:57.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149705 2026-02-19T16:24:57.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-19T16:24:57.192 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149705 2026-02-19T16:24:57.192 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:24:57.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-19T16:24:57.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:57.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888072 2026-02-19T16:24:57.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:57.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:24:57.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888072 2026-02-19T16:24:57.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:57.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888072 2026-02-19T16:24:57.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888072' 2026-02-19T16:24:57.418 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 146028888072 2026-02-19T16:24:57.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:24:57.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888073 -lt 146028888072 2026-02-19T16:24:57.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:57.647 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626440 2026-02-19T16:24:57.647 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:57.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:24:57.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626440 2026-02-19T16:24:57.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:57.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626440 2026-02-19T16:24:57.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626440' 2026-02-19T16:24:57.650 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 180388626440 2026-02-19T16:24:57.650 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:24:57.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626440 -lt 180388626440 2026-02-19T16:24:57.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:57.880 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364807 2026-02-19T16:24:57.880 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:57.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:24:57.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364807 2026-02-19T16:24:57.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:57.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364807 2026-02-19T16:24:57.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364807' 2026-02-19T16:24:57.883 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 214748364807 2026-02-19T16:24:57.883 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:24:58.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364808 -lt 214748364807 2026-02-19T16:24:58.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:24:58.118 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-283467841542 2026-02-19T16:24:58.118 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:24:58.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-19T16:24:58.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-283467841542 2026-02-19T16:24:58.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:24:58.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=283467841542 2026-02-19T16:24:58.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 283467841542' 2026-02-19T16:24:58.121 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.8 seq 283467841542 2026-02-19T16:24:58.121 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T16:24:58.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 283467841543 -lt 283467841542 2026-02-19T16:24:58.355 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:24:58.355 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:58.355 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:58.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:24:58.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:24:58.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:24:58.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:24:58.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:24:58.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:24:58.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:24:58.676 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:24:58.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:24:58.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:24:58.905 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:24:58.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:24:59.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-19T16:24:59.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:24:59.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:24:59.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:24:59.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:24:59.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:24:59.524 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:24:59.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:24:59.812 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:24:59.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:24:59.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:24:59.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:24:59.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:24:59.821 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:24:59.821 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:24:59.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:24:59.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:24:59.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:24:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:24:59.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:24:59.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:24:59.934 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:24:59.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:24:59.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:24:59.935 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:24:59.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:59.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:24:59.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:24:59.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:59.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:24:59.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:24:59.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:24:59.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:24:59.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:59.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:59.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:24:59.960 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:24:59.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:24:59.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:24:59.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:24:59.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:24:59.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:24:59.962 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:24:59.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:24:59.962 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:24:59.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:24:59.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:24:59.963 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:24:59.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:59.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:24:59.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:24:59.965 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:24:59.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:24:59.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:24:59.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:24:59.967 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:24:59.967 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:59.968 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:59.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:24:59.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:24:59.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:24:59.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:24:59.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:24:59.970 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:59.970 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:59.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:24:59.971 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:24:59.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:24:59.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:24:59.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:24:59.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:24:59.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:24:59.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:24:59.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:24:59.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:24:59.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:24:59.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:24:59.990 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:24:59.990 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:24:59.991 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:24:59.991 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:24:59.991 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:24:59.991 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:24:59.991 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:24:59.991 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:25:00.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:25:00.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:25:00.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:25:00.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:25:00.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:25:00.018 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:25:00.018 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:25:00.018 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:25:00.018 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:25:00.019 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:00.020 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:00.020 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:00.020 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:25:00.020 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:25:00.020 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:25:00.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:25:00.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:25:00.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:25:00.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:25:00.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:25:00.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:25:00.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:25:00.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:25:00.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:25:00.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:25:00.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:25:00.203 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:25:00.203 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:00.203 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:00.203 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:00.203 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:00.203 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:00.203 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:00.204 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:25:00.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:25:00.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:25:00.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:25:00.315 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:25:00.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:25:01.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:25:01.362 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:25:01.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:25:01.368 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:25:00.007+0000 7f9c16690c00 0 load: jerasure load: lrc 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T16:25:01.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:25:01.370 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T16:25:01.373 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:01.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:01.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:01.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:25:01.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:01.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f82ad484-f296-4df4-ac48-5d699e7e2dee 2026-02-19T16:25:01.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f82ad484-f296-4df4-ac48-5d699e7e2dee' 2026-02-19T16:25:01.377 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 f82ad484-f296-4df4-ac48-5d699e7e2dee 2026-02-19T16:25:01.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:01.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBdOZdp4GXZFhAA2o5iZUNqlyRaniaTEifLAw== 2026-02-19T16:25:01.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBdOZdp4GXZFhAA2o5iZUNqlyRaniaTEifLAw=="}' 2026-02-19T16:25:01.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f82ad484-f296-4df4-ac48-5d699e7e2dee -i td/test-erasure-eio/0/new.json 2026-02-19T16:25:01.488 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:01.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:25:01.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBdOZdp4GXZFhAA2o5iZUNqlyRaniaTEifLAw== --osd-uuid f82ad484-f296-4df4-ac48-5d699e7e2dee 2026-02-19T16:25:01.514 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:01.512+0000 7fbfc3834780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:01.515 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:01.514+0000 7fbfc3834780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:01.516 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:01.515+0000 7fbfc3834780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:01.517 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:01.515+0000 7fbfc3834780 -1 bdev(0x3074c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:01.517 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:01.515+0000 7fbfc3834780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:25:01.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:25:01.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:01.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:25:01.914 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:25:01.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:02.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:25:02.018 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:25:02.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:02.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:02.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:02.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:02.035 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:02.033+0000 7fc4c05d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:02.036 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:02.035+0000 7fc4c05d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:02.037 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:02.036+0000 7fc4c05d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:02.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:25:02.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:02.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:25:02.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:02.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:02.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:02.120 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:02.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:02.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:02.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:25:02.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:02.272 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:02.270+0000 7fc4c05d6780 -1 Falling back to public interface 2026-02-19T16:25:02.522 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:02.520+0000 7fc4c05d6780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:25:03.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:03.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:03.219 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:03.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:03.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:03.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:25:03.448 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3271724310,v1:127.0.0.1:6803/3271724310] [v2:127.0.0.1:6804/3271724310,v1:127.0.0.1:6805/3271724310] exists,up f82ad484-f296-4df4-ac48-5d699e7e2dee 2026-02-19T16:25:03.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:03.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:03.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:03.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:03.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:25:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=61dd6b9a-3bbf-4507-a80a-5718e2ba88c1 2026-02-19T16:25:03.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 61dd6b9a-3bbf-4507-a80a-5718e2ba88c1' 2026-02-19T16:25:03.456 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 61dd6b9a-3bbf-4507-a80a-5718e2ba88c1 2026-02-19T16:25:03.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:03.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBfOZdpscmLGxAAeXJWEERrB+SyRzQkF2fpeA== 2026-02-19T16:25:03.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBfOZdpscmLGxAAeXJWEERrB+SyRzQkF2fpeA=="}' 2026-02-19T16:25:03.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 61dd6b9a-3bbf-4507-a80a-5718e2ba88c1 -i td/test-erasure-eio/1/new.json 2026-02-19T16:25:03.697 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:03.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:25:03.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBfOZdpscmLGxAAeXJWEERrB+SyRzQkF2fpeA== --osd-uuid 61dd6b9a-3bbf-4507-a80a-5718e2ba88c1 2026-02-19T16:25:03.724 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:03.722+0000 7fd40a453780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:03.725 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:03.723+0000 7fd40a453780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:03.726 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:03.724+0000 7fd40a453780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:03.726 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:03.724+0000 7fd40a453780 -1 bdev(0x3d3cc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:03.726 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:03.724+0000 7fd40a453780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:25:04.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:25:04.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:04.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:25:04.048 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:25:04.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:04.372 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:25:04.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:25:04.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:04.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:04.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:04.373 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:04.390 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:04.388+0000 7fdee6030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:04.391 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:04.389+0000 7fdee6030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:04.392 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:04.390+0000 7fdee6030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:04.514 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:04.512+0000 7fdee6030780 -1 Falling back to public interface 2026-02-19T16:25:04.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:25:04.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:04.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:25:04.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:04.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:04.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:04.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:04.605 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:04.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:04.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:25:04.806 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:04.804+0000 7fdee6030780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:25:04.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:05.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:05.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:05.841 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:05.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:05.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:05.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:25:06.073 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1589253859,v1:127.0.0.1:6811/1589253859] [v2:127.0.0.1:6812/1589253859,v1:127.0.0.1:6813/1589253859] exists,up 61dd6b9a-3bbf-4507-a80a-5718e2ba88c1 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:06.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:06.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:06.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e5d669a-98ed-4300-9581-fd0a9e4c6bf3 2026-02-19T16:25:06.080 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 4e5d669a-98ed-4300-9581-fd0a9e4c6bf3 2026-02-19T16:25:06.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4e5d669a-98ed-4300-9581-fd0a9e4c6bf3' 2026-02-19T16:25:06.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:06.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBiOZdpED0vBRAASydmr+5R+JukW41nogF23A== 2026-02-19T16:25:06.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBiOZdpED0vBRAASydmr+5R+JukW41nogF23A=="}' 2026-02-19T16:25:06.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e5d669a-98ed-4300-9581-fd0a9e4c6bf3 -i td/test-erasure-eio/2/new.json 2026-02-19T16:25:06.324 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:25:06.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:25:06.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBiOZdpED0vBRAASydmr+5R+JukW41nogF23A== --osd-uuid 4e5d669a-98ed-4300-9581-fd0a9e4c6bf3 2026-02-19T16:25:06.351 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:06.349+0000 7ff1a5a3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:06.352 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:06.351+0000 7ff1a5a3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:06.353 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:06.351+0000 7ff1a5a3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:06.354 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:06.352+0000 7ff1a5a3b780 -1 bdev(0x2926c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:06.354 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:06.352+0000 7ff1a5a3b780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:25:06.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:25:06.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:06.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:25:06.671 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:25:06.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:06.990 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:25:06.991 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:25:06.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:06.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:06.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:06.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:07.008 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:07.006+0000 7f0cdf63b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:07.009 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:07.007+0000 7f0cdf63b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:07.010 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:07.008+0000 7f0cdf63b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:07.139 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:07.137+0000 7f0cdf63b780 -1 Falling back to public interface 2026-02-19T16:25:07.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:25:07.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:07.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:25:07.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:07.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:07.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:07.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:07.224 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:07.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:07.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:25:07.460 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:07.483 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:07.481+0000 7f0cdf63b780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:25:08.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:08.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:08.462 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:08.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:08.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:08.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:25:08.688 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4113059712,v1:127.0.0.1:6819/4113059712] [v2:127.0.0.1:6820/4113059712,v1:127.0.0.1:6821/4113059712] exists,up 4e5d669a-98ed-4300-9581-fd0a9e4c6bf3 2026-02-19T16:25:08.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:08.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:08.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:08.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:08.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:25:08.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:08.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:08.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:25:08.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:08.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:25:08.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:08.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:08.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:08.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:08.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:25:08.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:25:08.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:08.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:08.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:08.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:08.699 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:08.700 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:08.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:08.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:08.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:08.700 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:08.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:08.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:08.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:08.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:08.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:08.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:08.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:08.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:08.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:08.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:08.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:08.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:08.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:25:08.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:08.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eac752f6-5aa4-4ba4-b729-da5b790a8de1 2026-02-19T16:25:08.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 eac752f6-5aa4-4ba4-b729-da5b790a8de1' 2026-02-19T16:25:08.704 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 eac752f6-5aa4-4ba4-b729-da5b790a8de1 2026-02-19T16:25:08.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:08.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBkOZdp3pHWKRAAy626HkmT1qvTSyrxvAmiSg== 2026-02-19T16:25:08.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBkOZdp3pHWKRAAy626HkmT1qvTSyrxvAmiSg=="}' 2026-02-19T16:25:08.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eac752f6-5aa4-4ba4-b729-da5b790a8de1 -i td/test-erasure-eio/3/new.json 2026-02-19T16:25:08.936 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:25:08.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:25:08.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBkOZdp3pHWKRAAy626HkmT1qvTSyrxvAmiSg== --osd-uuid eac752f6-5aa4-4ba4-b729-da5b790a8de1 2026-02-19T16:25:08.963 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:08.961+0000 7f3c5d442780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:08.964 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:08.962+0000 7f3c5d442780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:08.965 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:08.963+0000 7f3c5d442780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:08.965 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:08.964+0000 7f3c5d442780 -1 bdev(0x3f40c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:08.965 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:08.964+0000 7f3c5d442780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:25:09.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:25:09.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:09.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:25:09.308 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:25:09.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:09.629 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:25:09.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:25:09.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:09.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:09.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:09.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:09.647 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:09.645+0000 7f174302d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:09.648 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:09.646+0000 7f174302d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:09.649 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:09.647+0000 7f174302d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:09.845 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:09.844+0000 7f174302d780 -1 Falling back to public interface 2026-02-19T16:25:09.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:25:09.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:09.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:25:09.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:09.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:09.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:09.857 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:09.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:09.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:09.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:25:10.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:10.119 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:10.116+0000 7f174302d780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:25:11.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:11.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:11.090 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:11.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:11.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:11.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:25:11.318 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2083017395,v1:127.0.0.1:6827/2083017395] [v2:127.0.0.1:6828/2083017395,v1:127.0.0.1:6829/2083017395] exists,up eac752f6-5aa4-4ba4-b729-da5b790a8de1 2026-02-19T16:25:11.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:11.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:11.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:11.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:11.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T16:25:11.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:11.324 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:11.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T16:25:11.326 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:11.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=68c38a73-d003-4479-8a76-589e5c8f17df 2026-02-19T16:25:11.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 68c38a73-d003-4479-8a76-589e5c8f17df' 2026-02-19T16:25:11.326 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 68c38a73-d003-4479-8a76-589e5c8f17df 2026-02-19T16:25:11.326 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:11.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBnOZdpCwnHExAApFzAGhWiIQ3XQRz6kWsBfA== 2026-02-19T16:25:11.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBnOZdpCwnHExAApFzAGhWiIQ3XQRz6kWsBfA=="}' 2026-02-19T16:25:11.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 68c38a73-d003-4479-8a76-589e5c8f17df -i td/test-erasure-eio/4/new.json 2026-02-19T16:25:11.570 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:25:11.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T16:25:11.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBnOZdpCwnHExAApFzAGhWiIQ3XQRz6kWsBfA== --osd-uuid 68c38a73-d003-4479-8a76-589e5c8f17df 2026-02-19T16:25:11.597 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:11.594+0000 7f346be5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:11.599 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:11.596+0000 7f346be5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:11.599 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:11.597+0000 7f346be5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:11.600 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:11.597+0000 7f346be5c780 -1 bdev(0x4398c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:11.600 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:11.597+0000 7f346be5c780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T16:25:11.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T16:25:11.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:11.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:25:11.936 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:25:11.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:12.261 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:25:12.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:25:12.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:12.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:12.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:12.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:12.278 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:12.275+0000 7f3fa8628780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:12.279 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:12.276+0000 7f3fa8628780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:12.280 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:12.277+0000 7f3fa8628780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:12.411 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:12.408+0000 7f3fa8628780 -1 Falling back to public interface 2026-02-19T16:25:12.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:25:12.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:12.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:25:12.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:12.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:12.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:12.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:12.488 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:12.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:12.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:25:12.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:12.745 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:12.742+0000 7f3fa8628780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:25:13.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:13.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:13.716 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:13.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:13.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:13.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:25:13.951 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1059119956,v1:127.0.0.1:6835/1059119956] [v2:127.0.0.1:6836/1059119956,v1:127.0.0.1:6837/1059119956] exists,up 68c38a73-d003-4479-8a76-589e5c8f17df 2026-02-19T16:25:13.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:13.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:13.956 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:13.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=91331ed5-c259-4830-9e44-24ed6d81377d 2026-02-19T16:25:13.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 91331ed5-c259-4830-9e44-24ed6d81377d' 2026-02-19T16:25:13.959 INFO:tasks.workunit.client.0.trial127.stdout:add osd5 91331ed5-c259-4830-9e44-24ed6d81377d 2026-02-19T16:25:13.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:13.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBpOZdprB53ORAAj1mCPNBLP3kyJW3xFTM1LQ== 2026-02-19T16:25:13.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBpOZdprB53ORAAj1mCPNBLP3kyJW3xFTM1LQ=="}' 2026-02-19T16:25:13.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 91331ed5-c259-4830-9e44-24ed6d81377d -i td/test-erasure-eio/5/new.json 2026-02-19T16:25:14.202 INFO:tasks.workunit.client.0.trial127.stdout:5 2026-02-19T16:25:14.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T16:25:14.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBpOZdprB53ORAAj1mCPNBLP3kyJW3xFTM1LQ== --osd-uuid 91331ed5-c259-4830-9e44-24ed6d81377d 2026-02-19T16:25:14.229 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.226+0000 7fbac7a5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:14.230 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.227+0000 7fbac7a5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:14.231 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.228+0000 7fbac7a5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:14.232 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.229+0000 7fbac7a5c780 -1 bdev(0x3f16c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:14.232 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.229+0000 7fbac7a5c780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T16:25:14.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T16:25:14.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:14.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T16:25:14.564 INFO:tasks.workunit.client.0.trial127.stdout:adding osd5 key to auth repository 2026-02-19T16:25:14.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:14.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T16:25:14.891 INFO:tasks.workunit.client.0.trial127.stdout:start osd.5 2026-02-19T16:25:14.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:14.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:14.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:14.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:14.908 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.905+0000 7fbcbc02e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:14.909 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.907+0000 7fbcbc02e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:14.910 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:14.907+0000 7fbcbc02e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:15.052 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:15.049+0000 7fbcbc02e780 -1 Falling back to public interface 2026-02-19T16:25:15.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T16:25:15.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:15.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T16:25:15.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:15.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:15.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:15.121 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:15.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:15.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:15.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:25:15.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:15.358 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:15.355+0000 7fbcbc02e780 -1 osd.5 0 log_to_monitors true 2026-02-19T16:25:16.350 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:16.350 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:16.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:16.351 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:16.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:16.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:25:16.583 INFO:tasks.workunit.client.0.trial127.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/68136893,v1:127.0.0.1:6843/68136893] [v2:127.0.0.1:6844/68136893,v1:127.0.0.1:6845/68136893] exists,up 91331ed5-c259-4830-9e44-24ed6d81377d 2026-02-19T16:25:16.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:16.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:16.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:16.589 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:16.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T16:25:16.591 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:16.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b037693c-6023-4588-984e-95edfd340b2c 2026-02-19T16:25:16.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 b037693c-6023-4588-984e-95edfd340b2c' 2026-02-19T16:25:16.591 INFO:tasks.workunit.client.0.trial127.stdout:add osd6 b037693c-6023-4588-984e-95edfd340b2c 2026-02-19T16:25:16.592 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:16.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBsOZdpoMKiIxAA/lZCiAP3E3e/ZUwciwB4gw== 2026-02-19T16:25:16.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBsOZdpoMKiIxAA/lZCiAP3E3e/ZUwciwB4gw=="}' 2026-02-19T16:25:16.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b037693c-6023-4588-984e-95edfd340b2c -i td/test-erasure-eio/6/new.json 2026-02-19T16:25:16.835 INFO:tasks.workunit.client.0.trial127.stdout:6 2026-02-19T16:25:16.842 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T16:25:16.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBsOZdpoMKiIxAA/lZCiAP3E3e/ZUwciwB4gw== --osd-uuid b037693c-6023-4588-984e-95edfd340b2c 2026-02-19T16:25:16.860 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:16.858+0000 7f53275fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:16.862 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:16.859+0000 7f53275fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:16.863 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:16.860+0000 7f53275fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:16.863 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:16.860+0000 7f53275fd780 -1 bdev(0x25d8c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:16.863 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:16.860+0000 7f53275fd780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T16:25:17.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T16:25:17.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:17.204 INFO:tasks.workunit.client.0.trial127.stdout:adding osd6 key to auth repository 2026-02-19T16:25:17.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T16:25:17.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:17.520 INFO:tasks.workunit.client.0.trial127.stdout:start osd.6 2026-02-19T16:25:17.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T16:25:17.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:17.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:17.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:17.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:17.537 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:17.535+0000 7f81ad33d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:17.539 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:17.536+0000 7f81ad33d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:17.539 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:17.537+0000 7f81ad33d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:17.672 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:17.669+0000 7f81ad33d780 -1 Falling back to public interface 2026-02-19T16:25:17.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T16:25:17.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:17.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T16:25:17.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:17.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:17.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:17.753 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:17.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:17.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:17.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:25:17.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:18.041 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:18.038+0000 7f81ad33d780 -1 osd.6 0 log_to_monitors true 2026-02-19T16:25:18.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:18.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:18.982 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:18.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:18.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:18.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:25:19.204 INFO:tasks.workunit.client.0.trial127.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/751572719,v1:127.0.0.1:6851/751572719] [v2:127.0.0.1:6852/751572719,v1:127.0.0.1:6853/751572719] exists,up b037693c-6023-4588-984e-95edfd340b2c 2026-02-19T16:25:19.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:19.205 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:25:19.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:25:19.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:25:19.250 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:25:19.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:25:19.256 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:25:02.272+0000 7fc4c05d6780 0 load: jerasure load: lrc 2026-02-19T16:25:19.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:25:19.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:25:19.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T16:25:19.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:25:19.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:25:19.915 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:25:19.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:25:20.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:25:20.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:25:20.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:25:20.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:25:20.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:25:20.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:25:20.927 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:25:20.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:25:20.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:25:20.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:25:21.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:25:21.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:25:21.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:25:21.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:25:21.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:25:21.013 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:25:21.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:25:21.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:21.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:25:21.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T16:25:21.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T16:25:21.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T16:25:21.300 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:21.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:25:21.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-19T16:25:21.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-19T16:25:21.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558' 2026-02-19T16:25:21.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:21.367 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:25:21.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T16:25:21.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T16:25:21.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333' 2026-02-19T16:25:21.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:21.431 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:25:21.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-19T16:25:21.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-19T16:25:21.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701' 2026-02-19T16:25:21.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:21.496 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:25:21.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-19T16:25:21.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-19T16:25:21.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-150323855364' 2026-02-19T16:25:21.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:21.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:25:21.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593733 2026-02-19T16:25:21.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593733 2026-02-19T16:25:21.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-150323855364 5-184683593733' 2026-02-19T16:25:21.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:21.625 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:25:21.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332099 2026-02-19T16:25:21.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332099 2026-02-19T16:25:21.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-150323855364 5-184683593733 6-219043332099' 2026-02-19T16:25:21.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:21.691 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-19T16:25:21.691 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:21.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:25:21.692 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T16:25:21.692 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:21.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-19T16:25:21.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T16:25:21.693 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803782 2026-02-19T16:25:21.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:25:21.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-19T16:25:21.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:25:22.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:25:22.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:25:23.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T16:25:23.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:23.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-19T16:25:23.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:23.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:25:23.156 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-19T16:25:23.157 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:23.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-19T16:25:23.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-19T16:25:23.158 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607558 2026-02-19T16:25:23.159 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:25:23.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-19T16:25:23.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:23.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-19T16:25:23.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:23.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:25:23.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T16:25:23.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:23.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-19T16:25:23.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T16:25:23.388 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411333 2026-02-19T16:25:23.388 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:25:23.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T16:25:23.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:23.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149701 2026-02-19T16:25:23.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:23.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:25:23.617 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-19T16:25:23.617 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:23.618 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149701 2026-02-19T16:25:23.618 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-19T16:25:23.619 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149701 2026-02-19T16:25:23.619 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:25:23.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-19T16:25:23.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:23.845 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855364 2026-02-19T16:25:23.845 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:23.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:25:23.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-19T16:25:23.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:23.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855364 2026-02-19T16:25:23.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-19T16:25:23.848 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 150323855364 2026-02-19T16:25:23.848 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:25:24.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-19T16:25:24.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:24.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593733 2026-02-19T16:25:24.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:24.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:25:24.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593733 2026-02-19T16:25:24.080 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:24.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593733 2026-02-19T16:25:24.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593733' 2026-02-19T16:25:24.081 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 184683593733 2026-02-19T16:25:24.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:25:24.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593733 -lt 184683593733 2026-02-19T16:25:24.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:24.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332099 2026-02-19T16:25:24.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:24.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:25:24.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332099 2026-02-19T16:25:24.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:24.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332099 2026-02-19T16:25:24.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332099' 2026-02-19T16:25:24.313 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 219043332099 2026-02-19T16:25:24.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:25:24.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332100 -lt 219043332099 2026-02-19T16:25:24.542 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:25:24.542 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:24.542 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:25:24.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:25:25.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:25:25.090 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:25:25.090 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:25.090 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:25.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:25:25.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:25:25.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:25:25.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-19T16:25:25.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-19T16:25:25.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:25:25.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:25:25.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-19T16:25:25.460 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-19T16:25:25.741 INFO:tasks.workunit.client.0.trial127.stderr:osd.0 is already out. 2026-02-19T16:25:25.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-19T16:25:25.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:25:25.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:25:25.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:25:25.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:25:25.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:25:25.752 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:25:25.753 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:25:25.753 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:25:25.753 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:25:25.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:25:25.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:25:25.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:25:25.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:25:25.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:25:25.839 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:26.061 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:25:26.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T16:25:26.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T16:25:26.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T16:25:26.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:26.127 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:25:26.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-19T16:25:26.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-19T16:25:26.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561' 2026-02-19T16:25:26.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:26.192 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:25:26.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-19T16:25:26.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-19T16:25:26.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336' 2026-02-19T16:25:26.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:26.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:25:26.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149704 2026-02-19T16:25:26.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149704 2026-02-19T16:25:26.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704' 2026-02-19T16:25:26.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:26.322 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:25:26.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855367 2026-02-19T16:25:26.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855367 2026-02-19T16:25:26.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704 4-150323855367' 2026-02-19T16:25:26.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:26.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:25:26.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593735 2026-02-19T16:25:26.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593735 2026-02-19T16:25:26.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704 4-150323855367 5-184683593735' 2026-02-19T16:25:26.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:26.450 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:25:26.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332102 2026-02-19T16:25:26.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332102 2026-02-19T16:25:26.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704 4-150323855367 5-184683593735 6-219043332102' 2026-02-19T16:25:26.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:26.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-19T16:25:26.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:26.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:25:26.515 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T16:25:26.515 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:26.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-19T16:25:26.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T16:25:26.517 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803785 2026-02-19T16:25:26.517 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:25:26.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T16:25:26.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:26.742 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607561 2026-02-19T16:25:26.743 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:26.744 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:25:26.744 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-19T16:25:26.744 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:26.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607561 2026-02-19T16:25:26.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-19T16:25:26.746 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607561 2026-02-19T16:25:26.746 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:25:26.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-19T16:25:26.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:26.976 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-19T16:25:26.976 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:26.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:25:26.978 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-19T16:25:26.978 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:26.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-19T16:25:26.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-19T16:25:26.979 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411336 2026-02-19T16:25:26.979 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:25:27.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411336 2026-02-19T16:25:27.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:25:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:25:28.208 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:25:28.434 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-19T16:25:28.434 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:28.435 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149704 2026-02-19T16:25:28.435 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:28.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:25:28.436 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149704 2026-02-19T16:25:28.436 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:28.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149704 2026-02-19T16:25:28.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149704' 2026-02-19T16:25:28.438 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149704 2026-02-19T16:25:28.438 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:25:28.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149704 -lt 111669149704 2026-02-19T16:25:28.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:28.667 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855367 2026-02-19T16:25:28.668 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:28.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:25:28.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855367 2026-02-19T16:25:28.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:28.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855367 2026-02-19T16:25:28.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855367' 2026-02-19T16:25:28.671 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 150323855367 2026-02-19T16:25:28.671 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:25:28.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855367 -lt 150323855367 2026-02-19T16:25:28.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:28.898 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593735 2026-02-19T16:25:28.898 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:28.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:25:28.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593735 2026-02-19T16:25:28.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:28.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593735 2026-02-19T16:25:28.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-19T16:25:28.901 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 184683593735 2026-02-19T16:25:28.901 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:25:29.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593736 -lt 184683593735 2026-02-19T16:25:29.128 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:29.128 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332102 2026-02-19T16:25:29.128 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:29.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:25:29.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332102 2026-02-19T16:25:29.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:29.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332102 2026-02-19T16:25:29.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332102' 2026-02-19T16:25:29.131 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 219043332102 2026-02-19T16:25:29.132 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:25:29.359 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332102 -lt 219043332102 2026-02-19T16:25:29.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:25:29.361 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:29.361 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:29.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:25:29.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:25:29.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:25:29.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:25:29.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:25:29.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:25:29.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:25:29.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:25:29.893 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=2 2026-02-19T16:25:29.894 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:25:29.895 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:29.895 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:30.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 = 5 2026-02-19T16:25:30.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 2 '!=' -1 2026-02-19T16:25:30.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T16:25:30.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=2 2026-02-19T16:25:30.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T16:25:30.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T16:25:30.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T16:25:30.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:25:30.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:25:30.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:25:30.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:25:30.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:25:30.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:25:30.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:25:30.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=2 2026-02-19T16:25:30.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:25:30.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:30.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:30.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 = 5 2026-02-19T16:25:30.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 2 '!=' 2 2026-02-19T16:25:30.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T16:25:30.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T16:25:30.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T16:25:30.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T16:25:30.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T16:25:30.860 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T16:25:30.860 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T16:25:31.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T16:25:31.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T16:25:31.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T16:25:31.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T16:25:31.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:25:31.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:25:31.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=2 2026-02-19T16:25:31.603 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:25:31.603 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:31.603 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:31.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 = 5 2026-02-19T16:25:31.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 2 '!=' 2 2026-02-19T16:25:31.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T16:25:31.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T16:25:31.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T16:25:31.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T16:25:31.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T16:25:31.919 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T16:25:31.919 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T16:25:32.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T16:25:32.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T16:25:32.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T16:25:32.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T16:25:32.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T16:25:32.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T16:25:32.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:25:32.638 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:25:32.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:25:32.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:25:32.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:25:32.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:25:32.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:25:32.861 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:25:32.861 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:25:32.861 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:32.862 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:33.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:25:33.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:25:33.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:25:33.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-19T16:25:33.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:25:33.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:25:33.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-19T16:25:33.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:25:33.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:25:33.179 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-19T16:25:33.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:25:33.196 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:25:33.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-19T16:25:33.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:25:33.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:25:33.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-19T16:25:33.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:25:33.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:25:33.198 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-19T16:25:33.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:25:33.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:25:33.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-19T16:25:33.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:25:33.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:25:33.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-19T16:25:33.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:25:33.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:25:33.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-19T16:25:33.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:25:33.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:25:33.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:25:33.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:25:33.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:25:33.477 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:25:33.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:25:33.765 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:25:33.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:25:33.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:25:33.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:25:33.884 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:25:33.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:25:33.885 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:25:33.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:25:33.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:25:33.887 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:25:33.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:25:33.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:25:33.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:25:33.888 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:25:33.889 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:25:33.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:25:33.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:25:33.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:25:33.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:33.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:33.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:25:33.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:25:33.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:25:33.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:25:33.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:25:33.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:25:33.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:25:33.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:25:33.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:25:33.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:25:33.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:25:33.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:25:33.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:25:33.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:25:33.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:25:33.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:25:33.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:25:33.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:25:33.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:25:33.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:25:33.911 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:25:33.912 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:25:33.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:25:33.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:25:33.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:25:33.913 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:25:33.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:25:33.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:25:33.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:25:33.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:25:33.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:33.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:33.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:25:33.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:25:33.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:25:33.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:25:33.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:25:33.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:33.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:33.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:25:33.919 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:25:33.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:25:33.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:25:33.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:25:33.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:25:33.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:25:33.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:25:33.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:25:33.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:25:33.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:25:33.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:25:33.938 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:25:33.939 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:33.939 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:33.939 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:33.939 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:33.939 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:33.939 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:33.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:25:33.963 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:25:33.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:25:33.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:25:33.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:25:33.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:25:33.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:25:33.966 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:25:34.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:25:34.010 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:25:34.010 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:25:34.010 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:25:34.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:25:34.011 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:25:34.011 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:25:34.011 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:25:34.011 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:34.012 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:34.012 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:34.012 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:25:34.012 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:25:34.012 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:25:34.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:25:34.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:25:34.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:25:34.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:25:34.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:25:34.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:25:34.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:25:34.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:25:34.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:34.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:34.150 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:34.150 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:34.150 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:34.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:34.151 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:25:34.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:25:34.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:25:34.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:25:34.261 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:25:34.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:25:35.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:25:35.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:25:35.270 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:25:35.270 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:35.270 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:35.271 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:35.271 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:25:35.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:25:35.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:25:35.308 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:25:35.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:25:35.314 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:25:33.954+0000 7f4764039c00 0 load: jerasure load: lrc 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T16:25:35.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:25:35.316 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T16:25:35.319 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:35.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:35.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:35.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:25:35.323 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:35.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c4dd7e0e-6e7a-4747-ae13-25876234f0c4 2026-02-19T16:25:35.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c4dd7e0e-6e7a-4747-ae13-25876234f0c4' 2026-02-19T16:25:35.323 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 c4dd7e0e-6e7a-4747-ae13-25876234f0c4 2026-02-19T16:25:35.323 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:35.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB/OZdpRROfExAA855mCuRbZ0Qj1MN5AuW0Sg== 2026-02-19T16:25:35.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB/OZdpRROfExAA855mCuRbZ0Qj1MN5AuW0Sg=="}' 2026-02-19T16:25:35.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c4dd7e0e-6e7a-4747-ae13-25876234f0c4 -i td/test-erasure-eio/0/new.json 2026-02-19T16:25:35.434 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:35.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:25:35.442 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB/OZdpRROfExAA855mCuRbZ0Qj1MN5AuW0Sg== --osd-uuid c4dd7e0e-6e7a-4747-ae13-25876234f0c4 2026-02-19T16:25:35.460 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:35.458+0000 7fe49d435780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:35.462 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:35.460+0000 7fe49d435780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:35.463 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:35.461+0000 7fe49d435780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:35.463 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:35.461+0000 7fe49d435780 -1 bdev(0x2cdcc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:35.463 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:35.461+0000 7fe49d435780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:25:35.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:25:35.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:35.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:25:35.909 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:25:35.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:36.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:25:36.014 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:25:36.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:36.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:36.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:36.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:36.030 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:36.029+0000 7f9b4e633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:36.032 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:36.030+0000 7f9b4e633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:36.033 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:36.031+0000 7f9b4e633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:36.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:25:36.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:36.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:25:36.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:36.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:36.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:36.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:36.114 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:36.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:36.114 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:25:36.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:36.255 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:36.253+0000 7f9b4e633780 -1 Falling back to public interface 2026-02-19T16:25:36.546 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:36.544+0000 7f9b4e633780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:25:37.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:37.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:37.211 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:37.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:37.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:37.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:25:37.442 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2068867015,v1:127.0.0.1:6803/2068867015] [v2:127.0.0.1:6804/2068867015,v1:127.0.0.1:6805/2068867015] exists,up c4dd7e0e-6e7a-4747-ae13-25876234f0c4 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:37.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:37.448 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cab77191-0d69-480a-b08c-038c04d17641 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 cab77191-0d69-480a-b08c-038c04d17641' 2026-02-19T16:25:37.450 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 cab77191-0d69-480a-b08c-038c04d17641 2026-02-19T16:25:37.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:37.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCBOZdpteAyGxAAKhzQNlk/4CMyzxDoar1Ykg== 2026-02-19T16:25:37.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCBOZdpteAyGxAAKhzQNlk/4CMyzxDoar1Ykg=="}' 2026-02-19T16:25:37.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cab77191-0d69-480a-b08c-038c04d17641 -i td/test-erasure-eio/1/new.json 2026-02-19T16:25:37.690 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:37.698 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:25:37.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCBOZdpteAyGxAAKhzQNlk/4CMyzxDoar1Ykg== --osd-uuid cab77191-0d69-480a-b08c-038c04d17641 2026-02-19T16:25:37.716 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:37.714+0000 7f2d8100a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:37.717 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:37.716+0000 7f2d8100a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:37.718 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:37.717+0000 7f2d8100a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:37.719 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:37.717+0000 7f2d8100a780 -1 bdev(0x399ec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:37.719 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:37.717+0000 7f2d8100a780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:25:38.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:25:38.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:38.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:25:38.041 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:25:38.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:38.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:25:38.361 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:25:38.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:38.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:38.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:38.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:38.377 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:38.375+0000 7f23fe63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:38.378 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:38.377+0000 7f23fe63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:38.379 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:38.378+0000 7f23fe63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:38.521 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:38.519+0000 7f23fe63a780 -1 Falling back to public interface 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:38.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:25:38.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:38.903 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:38.901+0000 7f23fe63a780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:25:39.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:39.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:39.823 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:39.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:39.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:39.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:25:40.051 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4206738287,v1:127.0.0.1:6811/4206738287] [v2:127.0.0.1:6812/4206738287,v1:127.0.0.1:6813/4206738287] exists,up cab77191-0d69-480a-b08c-038c04d17641 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:40.052 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:40.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:25:40.054 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:40.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8cd61d6e-d720-43df-ac82-f6356f0ad780 2026-02-19T16:25:40.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 8cd61d6e-d720-43df-ac82-f6356f0ad780' 2026-02-19T16:25:40.055 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 8cd61d6e-d720-43df-ac82-f6356f0ad780 2026-02-19T16:25:40.055 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:40.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEOZdp5E/nAxAADtkminKd1FGb0h1VuOHIrw== 2026-02-19T16:25:40.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEOZdp5E/nAxAADtkminKd1FGb0h1VuOHIrw=="}' 2026-02-19T16:25:40.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8cd61d6e-d720-43df-ac82-f6356f0ad780 -i td/test-erasure-eio/2/new.json 2026-02-19T16:25:40.299 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:25:40.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:25:40.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEOZdp5E/nAxAADtkminKd1FGb0h1VuOHIrw== --osd-uuid 8cd61d6e-d720-43df-ac82-f6356f0ad780 2026-02-19T16:25:40.327 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:40.325+0000 7ff26663c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:40.329 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:40.327+0000 7ff26663c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:40.329 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:40.328+0000 7ff26663c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:40.330 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:40.328+0000 7ff26663c780 -1 bdev(0x42f0c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:40.330 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:40.328+0000 7ff26663c780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:25:40.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:25:40.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:40.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:25:40.675 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:25:40.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:40.996 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:25:40.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:25:40.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:40.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:40.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:40.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:41.013 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:41.011+0000 7f411a026780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:41.015 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:41.013+0000 7f411a026780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:41.016 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:41.014+0000 7f411a026780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:41.199 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:41.197+0000 7f411a026780 -1 Falling back to public interface 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:41.222 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:41.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:41.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:25:41.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:41.550 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:41.548+0000 7f411a026780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:25:42.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:42.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:42.450 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:42.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:42.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:42.451 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:25:42.679 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3453035958,v1:127.0.0.1:6819/3453035958] [v2:127.0.0.1:6820/3453035958,v1:127.0.0.1:6821/3453035958] exists,up 8cd61d6e-d720-43df-ac82-f6356f0ad780 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:25:42.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:42.684 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:42.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b70b62c3-2ee5-4f62-87a0-8f31ace3a105 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 b70b62c3-2ee5-4f62-87a0-8f31ace3a105' 2026-02-19T16:25:42.686 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 b70b62c3-2ee5-4f62-87a0-8f31ace3a105 2026-02-19T16:25:42.687 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:42.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCGOZdpk3M/KRAAEh+iVojlT/MxQTkuU54KaQ== 2026-02-19T16:25:42.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCGOZdpk3M/KRAAEh+iVojlT/MxQTkuU54KaQ=="}' 2026-02-19T16:25:42.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b70b62c3-2ee5-4f62-87a0-8f31ace3a105 -i td/test-erasure-eio/3/new.json 2026-02-19T16:25:42.925 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:25:42.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:25:42.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGOZdpk3M/KRAAEh+iVojlT/MxQTkuU54KaQ== --osd-uuid b70b62c3-2ee5-4f62-87a0-8f31ace3a105 2026-02-19T16:25:42.954 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:42.952+0000 7feb0d036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:42.955 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:42.953+0000 7feb0d036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:42.956 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:42.954+0000 7feb0d036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:42.956 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:42.955+0000 7feb0d036780 -1 bdev(0x2932c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:42.957 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:42.955+0000 7feb0d036780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:25:43.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:25:43.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:43.294 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:25:43.294 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:25:43.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:43.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:25:43.615 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:25:43.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:43.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:43.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:43.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:43.632 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:43.630+0000 7f26e7835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:43.634 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:43.631+0000 7f26e7835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:43.635 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:43.632+0000 7f26e7835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:43.759 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:43.756+0000 7f26e7835780 -1 Falling back to public interface 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:43.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:25:44.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:44.089 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:44.086+0000 7f26e7835780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:25:45.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:45.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:45.079 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:45.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:45.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:45.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:25:45.315 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3983395951,v1:127.0.0.1:6827/3983395951] [v2:127.0.0.1:6828/3983395951,v1:127.0.0.1:6829/3983395951] exists,up b70b62c3-2ee5-4f62-87a0-8f31ace3a105 2026-02-19T16:25:45.319 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:45.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:45.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:45.322 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=efc443d2-6b0b-4755-8b3d-536d44e5156d 2026-02-19T16:25:45.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 efc443d2-6b0b-4755-8b3d-536d44e5156d' 2026-02-19T16:25:45.324 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 efc443d2-6b0b-4755-8b3d-536d44e5156d 2026-02-19T16:25:45.324 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:45.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJOZdpNuafExAAUPiccvwhkO5r8v7ANAizJg== 2026-02-19T16:25:45.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJOZdpNuafExAAUPiccvwhkO5r8v7ANAizJg=="}' 2026-02-19T16:25:45.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new efc443d2-6b0b-4755-8b3d-536d44e5156d -i td/test-erasure-eio/4/new.json 2026-02-19T16:25:45.563 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:25:45.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T16:25:45.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJOZdpNuafExAAUPiccvwhkO5r8v7ANAizJg== --osd-uuid efc443d2-6b0b-4755-8b3d-536d44e5156d 2026-02-19T16:25:45.590 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:45.587+0000 7f66f19d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:45.591 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:45.589+0000 7f66f19d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:45.592 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:45.589+0000 7f66f19d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:45.593 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:45.590+0000 7f66f19d6780 -1 bdev(0x2888c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:45.593 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:45.590+0000 7f66f19d6780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T16:25:45.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T16:25:45.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:45.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:25:45.920 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:25:45.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:46.237 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:25:46.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:25:46.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:46.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:46.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:46.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:46.255 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:46.252+0000 7f0f05c56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:46.257 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:46.254+0000 7f0f05c56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:46.258 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:46.255+0000 7f0f05c56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:46.391 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:46.388+0000 7f0f05c56780 -1 Falling back to public interface 2026-02-19T16:25:46.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:25:46.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:46.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:25:46.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:46.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:46.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:46.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:46.466 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:46.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:46.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:25:46.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:46.723 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:46.720+0000 7f0f05c56780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:25:47.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:47.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:47.693 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:47.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:47.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:47.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:25:47.924 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/4056135966,v1:127.0.0.1:6835/4056135966] [v2:127.0.0.1:6836/4056135966,v1:127.0.0.1:6837/4056135966] exists,up efc443d2-6b0b-4755-8b3d-536d44e5156d 2026-02-19T16:25:47.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:47.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:47.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:47.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:47.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T16:25:47.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T16:25:47.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:47.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=40f8cdd4-b360-4ad5-9b0c-7c7d69ef6366 2026-02-19T16:25:47.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 40f8cdd4-b360-4ad5-9b0c-7c7d69ef6366' 2026-02-19T16:25:47.932 INFO:tasks.workunit.client.0.trial127.stdout:add osd5 40f8cdd4-b360-4ad5-9b0c-7c7d69ef6366 2026-02-19T16:25:47.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:47.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCLOZdp8MncNxAAARY++626bO1uzKPMUUFedg== 2026-02-19T16:25:47.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCLOZdp8MncNxAAARY++626bO1uzKPMUUFedg=="}' 2026-02-19T16:25:47.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 40f8cdd4-b360-4ad5-9b0c-7c7d69ef6366 -i td/test-erasure-eio/5/new.json 2026-02-19T16:25:48.172 INFO:tasks.workunit.client.0.trial127.stdout:5 2026-02-19T16:25:48.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T16:25:48.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCLOZdp8MncNxAAARY++626bO1uzKPMUUFedg== --osd-uuid 40f8cdd4-b360-4ad5-9b0c-7c7d69ef6366 2026-02-19T16:25:48.201 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.199+0000 7fb528c3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:48.203 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.200+0000 7fb528c3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:48.204 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.201+0000 7fb528c3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:48.204 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.201+0000 7fb528c3d780 -1 bdev(0x3848c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:48.204 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.202+0000 7fb528c3d780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T16:25:48.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T16:25:48.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:48.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T16:25:48.525 INFO:tasks.workunit.client.0.trial127.stdout:adding osd5 key to auth repository 2026-02-19T16:25:48.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:48.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T16:25:48.843 INFO:tasks.workunit.client.0.trial127.stdout:start osd.5 2026-02-19T16:25:48.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:48.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:48.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:48.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:48.861 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.858+0000 7f8b61a53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:48.862 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.860+0000 7f8b61a53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:48.863 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:48.861+0000 7f8b61a53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:49.035 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:49.031+0000 7f8b61a53780 -1 Falling back to public interface 2026-02-19T16:25:49.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T16:25:49.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:49.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T16:25:49.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:49.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:49.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:49.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:49.073 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:49.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:49.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:25:49.298 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:49.396 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:49.394+0000 7f8b61a53780 -1 osd.5 0 log_to_monitors true 2026-02-19T16:25:50.299 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:50.299 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:50.300 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:50.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:50.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:50.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:25:50.526 INFO:tasks.workunit.client.0.trial127.stdout:osd.5 up in weight 1 up_from 41 up_thru 42 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1450686378,v1:127.0.0.1:6843/1450686378] [v2:127.0.0.1:6844/1450686378,v1:127.0.0.1:6845/1450686378] exists,up 40f8cdd4-b360-4ad5-9b0c-7c7d69ef6366 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:25:50.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:50.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:25:50.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:25:50.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:25:50.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T16:25:50.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:25:50.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=76ef75a4-cd0b-4210-93be-e17945573a5d 2026-02-19T16:25:50.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 76ef75a4-cd0b-4210-93be-e17945573a5d' 2026-02-19T16:25:50.530 INFO:tasks.workunit.client.0.trial127.stdout:add osd6 76ef75a4-cd0b-4210-93be-e17945573a5d 2026-02-19T16:25:50.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:25:50.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCOOZdpOxczIBAAvTHk4gXsop/SfPfSfm8CTg== 2026-02-19T16:25:50.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCOOZdpOxczIBAAvTHk4gXsop/SfPfSfm8CTg=="}' 2026-02-19T16:25:50.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 76ef75a4-cd0b-4210-93be-e17945573a5d -i td/test-erasure-eio/6/new.json 2026-02-19T16:25:50.777 INFO:tasks.workunit.client.0.trial127.stdout:6 2026-02-19T16:25:50.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T16:25:50.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCOOZdpOxczIBAAvTHk4gXsop/SfPfSfm8CTg== --osd-uuid 76ef75a4-cd0b-4210-93be-e17945573a5d 2026-02-19T16:25:50.804 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:50.802+0000 7f49bb429780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:50.806 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:50.803+0000 7f49bb429780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:50.807 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:50.804+0000 7f49bb429780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:50.807 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:50.804+0000 7f49bb429780 -1 bdev(0x2484c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T16:25:50.807 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:50.805+0000 7f49bb429780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T16:25:51.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T16:25:51.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:25:51.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T16:25:51.124 INFO:tasks.workunit.client.0.trial127.stdout:adding osd6 key to auth repository 2026-02-19T16:25:51.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:25:51.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T16:25:51.446 INFO:tasks.workunit.client.0.trial127.stdout:start osd.6 2026-02-19T16:25:51.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:25:51.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:25:51.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:25:51.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:25:51.463 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:51.460+0000 7f8e4b60b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:51.464 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:51.461+0000 7f8e4b60b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:51.465 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:51.462+0000 7f8e4b60b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:25:51.609 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:51.606+0000 7f8e4b60b780 -1 Falling back to public interface 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:51.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:25:51.905 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:25:51.934 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:25:51.931+0000 7f8e4b60b780 -1 osd.6 0 log_to_monitors true 2026-02-19T16:25:52.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:25:52.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:25:52.906 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:25:52.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:25:52.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:25:52.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:25:53.143 INFO:tasks.workunit.client.0.trial127.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/110324029,v1:127.0.0.1:6851/110324029] [v2:127.0.0.1:6852/110324029,v1:127.0.0.1:6853/110324029] exists,up 76ef75a4-cd0b-4210-93be-e17945573a5d 2026-02-19T16:25:53.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:53.144 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:25:53.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:25:53.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:25:53.190 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:25:53.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:25:53.196 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:25:36.256+0000 7f9b4e633780 0 load: jerasure load: lrc 2026-02-19T16:25:53.196 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:25:53.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T16:25:53.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:25:53.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:25:53.861 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:25:53.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:25:54.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:25:54.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:25:54.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:25:54.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:25:54.871 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:25:54.871 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:25:54.871 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:25:54.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:25:54.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:25:54.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:25:54.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:25:54.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:25:54.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:25:54.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:25:54.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:25:54.960 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:25:55.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:55.187 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:25:55.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T16:25:55.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T16:25:55.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T16:25:55.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:55.251 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:25:55.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-19T16:25:55.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-19T16:25:55.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558' 2026-02-19T16:25:55.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:55.315 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:25:55.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T16:25:55.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T16:25:55.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333' 2026-02-19T16:25:55.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:55.382 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:25:55.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182405 2026-02-19T16:25:55.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182405 2026-02-19T16:25:55.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182405' 2026-02-19T16:25:55.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:55.446 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:25:55.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920772 2026-02-19T16:25:55.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920772 2026-02-19T16:25:55.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182405 4-141733920772' 2026-02-19T16:25:55.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:55.511 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:25:55.576 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659140 2026-02-19T16:25:55.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659140 2026-02-19T16:25:55.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182405 4-141733920772 5-176093659140' 2026-02-19T16:25:55.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:25:55.577 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:25:55.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-19T16:25:55.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-19T16:25:55.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182405 4-141733920772 5-176093659140 6-210453397507' 2026-02-19T16:25:55.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:55.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-19T16:25:55.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:55.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:25:55.642 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T16:25:55.642 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:55.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-19T16:25:55.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T16:25:55.643 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803782 2026-02-19T16:25:55.644 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:25:55.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-19T16:25:55.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:25:56.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:25:56.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:25:57.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T16:25:57.097 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:57.097 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-19T16:25:57.098 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:57.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:25:57.099 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-19T16:25:57.099 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:57.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-19T16:25:57.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-19T16:25:57.100 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607558 2026-02-19T16:25:57.100 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:25:57.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-19T16:25:57.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:57.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-19T16:25:57.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:57.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:25:57.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T16:25:57.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:57.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-19T16:25:57.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T16:25:57.334 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411333 2026-02-19T16:25:57.334 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:25:57.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T16:25:57.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:57.564 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182405 2026-02-19T16:25:57.564 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:57.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:25:57.565 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182405 2026-02-19T16:25:57.565 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:57.566 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182405 2026-02-19T16:25:57.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-19T16:25:57.567 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 107374182405 2026-02-19T16:25:57.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:25:57.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-19T16:25:57.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:57.797 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920772 2026-02-19T16:25:57.797 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:57.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:25:57.798 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920772 2026-02-19T16:25:57.798 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:57.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920772 2026-02-19T16:25:57.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920772' 2026-02-19T16:25:57.800 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 141733920772 2026-02-19T16:25:57.800 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:25:58.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920772 -lt 141733920772 2026-02-19T16:25:58.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:58.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-176093659140 2026-02-19T16:25:58.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:58.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:25:58.034 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-176093659140 2026-02-19T16:25:58.034 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:58.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659140 2026-02-19T16:25:58.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 176093659140' 2026-02-19T16:25:58.035 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 176093659140 2026-02-19T16:25:58.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:25:58.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659140 -lt 176093659140 2026-02-19T16:25:58.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:25:58.265 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397507 2026-02-19T16:25:58.265 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:25:58.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:25:58.266 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397507 2026-02-19T16:25:58.266 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:25:58.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397507 2026-02-19T16:25:58.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-19T16:25:58.268 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 210453397507 2026-02-19T16:25:58.268 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:25:58.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397508 -lt 210453397507 2026-02-19T16:25:58.497 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:25:58.497 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:58.498 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:58.807 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:25:58.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:25:58.808 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:25:58.808 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:25:58.808 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:25:58.809 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:25:58.809 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:25:58.809 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:25:59.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:25:59.029 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:25:59.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:25:59.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:25:59.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:25:59.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:25:59.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:25:59.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-19T16:25:59.346 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:25:59.347 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-19T16:25:59.364 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:25:59.365 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:25:59.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-19T16:25:59.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-19T16:25:59.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:25:59.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:25:59.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-19T16:25:59.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:25:59.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:25:59.402 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:25:59.633 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:25:59.633 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:25:59.634 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T16:25:59.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:25:59.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:59.637 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:25:59.638 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:25:59.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:25:59.686 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:25:59.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-19T16:25:59.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-19T16:26:00.013 INFO:tasks.workunit.client.0.trial127.stderr:osd.0 is already out. 2026-02-19T16:26:00.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-19T16:26:00.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:26:00.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:26:00.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:26:00.023 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:26:00.023 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:26:00.024 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:26:00.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:26:00.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:26:00.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:26:00.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:26:00.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:26:00.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:26:00.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:26:00.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:26:00.108 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:26:00.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:26:00.335 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:26:00.335 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:26:00.335 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:26:00.335 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:26:00.336 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:26:00.336 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:26:00.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:26:00.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:00.336 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:26:00.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T16:26:00.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T16:26:00.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T16:26:00.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:00.400 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:26:00.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-19T16:26:00.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-19T16:26:00.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561' 2026-02-19T16:26:00.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:00.463 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:26:00.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-19T16:26:00.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-19T16:26:00.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336' 2026-02-19T16:26:00.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:00.529 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:26:00.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182408 2026-02-19T16:26:00.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182408 2026-02-19T16:26:00.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-107374182408' 2026-02-19T16:26:00.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:00.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:26:00.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920775 2026-02-19T16:26:00.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920775 2026-02-19T16:26:00.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-107374182408 4-141733920775' 2026-02-19T16:26:00.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:00.659 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:26:00.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659143 2026-02-19T16:26:00.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659143 2026-02-19T16:26:00.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-107374182408 4-141733920775 5-176093659143' 2026-02-19T16:26:00.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:00.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:26:00.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-19T16:26:00.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-19T16:26:00.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-107374182408 4-141733920775 5-176093659143 6-210453397510' 2026-02-19T16:26:00.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:00.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-19T16:26:00.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:00.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:26:00.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T16:26:00.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:00.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-19T16:26:00.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T16:26:00.792 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803785 2026-02-19T16:26:00.793 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:26:01.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 0 -lt 25769803785 2026-02-19T16:26:01.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:26:02.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:26:02.027 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:26:02.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T16:26:02.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:02.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607561 2026-02-19T16:26:02.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:02.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:26:02.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-19T16:26:02.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:02.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607561 2026-02-19T16:26:02.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-19T16:26:02.260 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607561 2026-02-19T16:26:02.260 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:26:02.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-19T16:26:02.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:02.489 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-19T16:26:02.489 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:02.490 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:26:02.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-19T16:26:02.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:02.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-19T16:26:02.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-19T16:26:02.493 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411336 2026-02-19T16:26:02.493 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:26:02.722 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-19T16:26:02.723 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:02.723 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182408 2026-02-19T16:26:02.723 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:02.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:26:02.724 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182408 2026-02-19T16:26:02.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:02.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182408 2026-02-19T16:26:02.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182408' 2026-02-19T16:26:02.726 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 107374182408 2026-02-19T16:26:02.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:26:02.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182408 -lt 107374182408 2026-02-19T16:26:02.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:02.955 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920775 2026-02-19T16:26:02.955 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:02.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:26:02.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920775 2026-02-19T16:26:02.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:02.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920775 2026-02-19T16:26:02.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920775' 2026-02-19T16:26:02.958 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 141733920775 2026-02-19T16:26:02.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:26:03.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920775 -lt 141733920775 2026-02-19T16:26:03.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:03.194 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-176093659143 2026-02-19T16:26:03.195 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:03.195 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:26:03.196 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-176093659143 2026-02-19T16:26:03.196 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:03.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659143 2026-02-19T16:26:03.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 176093659143' 2026-02-19T16:26:03.197 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 176093659143 2026-02-19T16:26:03.197 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:26:03.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659143 -lt 176093659143 2026-02-19T16:26:03.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:03.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397510 2026-02-19T16:26:03.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:03.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:26:03.429 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397510 2026-02-19T16:26:03.429 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:03.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397510 2026-02-19T16:26:03.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397510' 2026-02-19T16:26:03.430 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 210453397510 2026-02-19T16:26:03.431 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:26:03.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-19T16:26:03.662 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:26:03.662 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:03.662 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:03.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:26:03.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:26:03.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:26:03.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:26:03.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:26:03.982 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:26:03.982 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:26:03.982 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:26:04.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T16:26:04.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:26:04.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:04.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:04.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T16:26:04.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T16:26:04.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T16:26:04.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T16:26:04.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T16:26:04.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T16:26:04.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T16:26:04.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:26:04.623 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:26:04.623 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:26:04.623 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:26:04.623 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:26:04.624 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:26:04.624 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:26:04.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T16:26:04.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:26:04.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:04.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:05.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T16:26:05.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T16:26:05.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T16:26:05.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T16:26:05.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T16:26:05.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T16:26:05.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T16:26:05.164 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T16:26:05.164 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T16:26:05.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T16:26:05.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T16:26:05.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T16:26:05.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T16:26:05.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T16:26:05.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T16:26:05.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:26:05.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:26:05.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:26:05.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:26:05.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:26:05.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:26:05.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:26:05.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T16:26:05.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:26:05.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:05.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:06.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T16:26:06.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T16:26:06.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T16:26:06.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T16:26:06.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T16:26:06.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T16:26:06.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T16:26:06.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T16:26:06.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T16:26:06.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T16:26:06.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T16:26:06.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T16:26:06.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T16:26:06.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T16:26:06.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T16:26:06.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:26:06.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:26:06.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:26:06.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:26:06.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:26:06.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:26:06.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:26:07.180 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:26:07.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:26:07.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:07.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:07.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:26:07.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:26:07.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:26:07.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-19T16:26:07.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:26:07.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:26:07.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-19T16:26:07.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:26:07.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:26:07.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-19T16:26:07.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:26:07.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:26:07.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-19T16:26:07.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:26:07.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:26:07.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-19T16:26:07.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:26:07.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:26:07.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-19T16:26:07.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:26:07.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:26:07.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-19T16:26:07.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:26:07.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:26:07.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-19T16:26:07.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:26:07.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:26:07.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-19T16:26:07.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:26:07.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:26:07.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:26:07.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:26:07.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:26:07.830 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:26:07.839 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:26:08.117 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:26:08.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:26:08.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:26:08.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:26:08.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:26:08.127 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:26:08.127 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:26:08.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:26:08.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:26:08.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:26:08.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:26:08.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:26:08.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:26:08.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:26:08.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:26:08.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:26:08.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:26:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:26:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:26:08.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:26:08.243 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:26:08.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:26:08.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:26:08.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:26:08.261 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:26:08.261 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:08.261 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:08.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:26:08.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:26:08.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:26:08.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:26:08.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:26:08.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:26:08.265 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:26:08.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:26:08.266 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:26:08.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:26:08.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:26:08.268 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:26:08.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:26:08.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:26:08.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:26:08.269 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:26:08.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:26:08.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:26:08.270 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:26:08.271 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:26:08.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:08.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:08.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:26:08.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:26:08.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:26:08.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:26:08.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:26:08.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:08.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:08.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:26:08.275 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:26:08.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:26:08.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:26:08.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:26:08.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:26:08.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:26:08.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:26:08.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:26:08.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:26:08.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:26:08.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:26:08.295 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:26:08.295 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:08.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:08.296 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:08.296 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:08.296 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:08.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:08.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:26:08.319 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:26:08.320 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:26:08.320 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:26:08.320 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:26:08.320 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:26:08.322 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:26:08.323 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:26:08.364 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:26:08.364 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:26:08.364 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:26:08.364 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:26:08.364 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:26:08.364 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:26:08.364 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:26:08.365 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:26:08.365 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:08.366 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:08.366 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:08.366 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:26:08.366 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:26:08.366 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:26:08.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:26:08.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:26:08.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:26:08.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:26:08.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:26:08.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:26:08.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:26:08.508 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:26:08.508 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:08.509 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:08.509 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:08.509 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:08.509 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:08.509 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:08.509 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:26:08.510 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:26:08.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:26:08.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:26:08.619 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:26:08.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:26:09.628 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:26:09.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:26:09.668 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:26:09.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:26:09.674 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:26:08.311+0000 7f3456034c00 0 load: jerasure load: lrc 2026-02-19T16:26:09.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-19T16:26:09.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-19T16:26:09.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:26:09.676 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-19T16:26:09.681 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-19T16:26:09.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:26:09.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:26:09.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:09.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:26:09.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=32443330-00dd-4f7c-9657-ac9884140343 2026-02-19T16:26:09.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 32443330-00dd-4f7c-9657-ac9884140343' 2026-02-19T16:26:09.685 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 32443330-00dd-4f7c-9657-ac9884140343 2026-02-19T16:26:09.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:26:09.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQChOZdpUrAuKRAA7vrshiV/dJk+Osy9yz1CUw== 2026-02-19T16:26:09.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQChOZdpUrAuKRAA7vrshiV/dJk+Osy9yz1CUw=="}' 2026-02-19T16:26:09.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 32443330-00dd-4f7c-9657-ac9884140343 -i td/test-erasure-eio/0/new.json 2026-02-19T16:26:09.799 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:26:09.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:26:09.807 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChOZdpUrAuKRAA7vrshiV/dJk+Osy9yz1CUw== --osd-uuid 32443330-00dd-4f7c-9657-ac9884140343 2026-02-19T16:26:09.828 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:09.826+0000 7fef7f432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:09.830 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:09.828+0000 7fef7f432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:09.831 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:09.829+0000 7fef7f432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:09.831 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:09.829+0000 7fef7f432780 -1 bdev(0x2c80c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:26:09.831 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:09.829+0000 7fef7f432780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:26:10.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:26:10.215 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:26:10.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:26:10.216 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:26:10.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:26:10.323 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:26:10.324 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:26:10.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:26:10.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:26:10.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:26:10.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:26:10.343 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:10.342+0000 7fb1f5410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:10.345 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:10.343+0000 7fb1f5410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:10.346 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:10.344+0000 7fb1f5410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:10.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:26:10.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:26:10.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:26:10.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:26:10.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:26:10.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:10.431 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:26:10.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:26:10.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:10.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:26:10.525 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:10.523+0000 7fb1f5410780 -1 Falling back to public interface 2026-02-19T16:26:10.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:26:10.793 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:10.791+0000 7fb1f5410780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:26:11.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:26:11.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:11.531 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:26:11.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:26:11.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:11.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1638871165,v1:127.0.0.1:6801/1638871165] [v2:127.0.0.1:6802/1638871165,v1:127.0.0.1:6803/1638871165] exists,up 32443330-00dd-4f7c-9657-ac9884140343 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:26:11.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:11.766 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:26:11.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:26:11.768 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:26:11.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=32c94a37-5880-46d0-8139-468175ffa89f 2026-02-19T16:26:11.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 32c94a37-5880-46d0-8139-468175ffa89f' 2026-02-19T16:26:11.768 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 32c94a37-5880-46d0-8139-468175ffa89f 2026-02-19T16:26:11.769 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:26:11.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCjOZdpdyF9LhAA6+lN19LvZxunEbH8kMK+Jw== 2026-02-19T16:26:11.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCjOZdpdyF9LhAA6+lN19LvZxunEbH8kMK+Jw=="}' 2026-02-19T16:26:11.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 32c94a37-5880-46d0-8139-468175ffa89f -i td/test-erasure-eio/1/new.json 2026-02-19T16:26:12.020 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:26:12.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:26:12.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCjOZdpdyF9LhAA6+lN19LvZxunEbH8kMK+Jw== --osd-uuid 32c94a37-5880-46d0-8139-468175ffa89f 2026-02-19T16:26:12.050 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.048+0000 7f5126c5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:12.051 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.049+0000 7f5126c5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:12.052 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.050+0000 7f5126c5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:12.052 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.050+0000 7f5126c5a780 -1 bdev(0x2aeec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:26:12.052 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.051+0000 7f5126c5a780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:26:12.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:26:12.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:26:12.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:26:12.377 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:26:12.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:26:12.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:26:12.703 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:26:12.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:26:12.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:26:12.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:26:12.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:26:12.722 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.721+0000 7f7bcfa26780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:12.724 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.722+0000 7f7bcfa26780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:12.725 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.723+0000 7f7bcfa26780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:12.864 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:12.862+0000 7f7bcfa26780 -1 Falling back to public interface 2026-02-19T16:26:12.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:26:12.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:26:12.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:26:12.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:26:12.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:26:12.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:12.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:26:12.941 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:26:12.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:12.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:26:13.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:26:13.200 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:13.198+0000 7f7bcfa26780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:26:14.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:26:14.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:14.175 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:26:14.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:26:14.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:14.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:26:14.409 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3224989036,v1:127.0.0.1:6811/3224989036] [v2:127.0.0.1:6812/3224989036,v1:127.0.0.1:6813/3224989036] exists,up 32c94a37-5880-46d0-8139-468175ffa89f 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:26:14.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:26:14.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:26:14.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2b1aec2a-b43a-4f83-bddc-edb82f528dc1 2026-02-19T16:26:14.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2b1aec2a-b43a-4f83-bddc-edb82f528dc1' 2026-02-19T16:26:14.417 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 2b1aec2a-b43a-4f83-bddc-edb82f528dc1 2026-02-19T16:26:14.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:26:14.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmOZdpw9ZKGRAA0QPW/hvS+D7ocKXktLFGVQ== 2026-02-19T16:26:14.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmOZdpw9ZKGRAA0QPW/hvS+D7ocKXktLFGVQ=="}' 2026-02-19T16:26:14.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2b1aec2a-b43a-4f83-bddc-edb82f528dc1 -i td/test-erasure-eio/2/new.json 2026-02-19T16:26:14.661 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:26:14.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:26:14.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmOZdpw9ZKGRAA0QPW/hvS+D7ocKXktLFGVQ== --osd-uuid 2b1aec2a-b43a-4f83-bddc-edb82f528dc1 2026-02-19T16:26:14.690 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:14.688+0000 7fa80262f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:14.692 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:14.690+0000 7fa80262f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:14.693 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:14.691+0000 7fa80262f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:14.693 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:14.691+0000 7fa80262f780 -1 bdev(0x259ac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:26:14.694 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:14.691+0000 7fa80262f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:26:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:26:15.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:26:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:26:15.016 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:26:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:26:15.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:26:15.336 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:26:15.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:26:15.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:26:15.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:26:15.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:26:15.355 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:15.354+0000 7f31bf442780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:15.357 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:15.355+0000 7f31bf442780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:15.358 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:15.356+0000 7f31bf442780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:15.475 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:15.474+0000 7f31bf442780 -1 Falling back to public interface 2026-02-19T16:26:15.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:26:15.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:26:15.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:26:15.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:26:15.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:26:15.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:15.562 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:26:15.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:26:15.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:15.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:26:15.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:26:15.837 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:15.835+0000 7f31bf442780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:26:16.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:26:16.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:16.790 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:26:16.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:26:16.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:16.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:26:17.022 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1667008267,v1:127.0.0.1:6819/1667008267] [v2:127.0.0.1:6820/1667008267,v1:127.0.0.1:6821/1667008267] exists,up 2b1aec2a-b43a-4f83-bddc-edb82f528dc1 2026-02-19T16:26:17.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:26:17.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:26:17.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:26:17.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:26:17.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:26:17.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:26:17.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:17.025 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:26:17.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:26:17.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:26:17.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:26:17.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:26:17.027 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:26:17.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e4e3f6dc-8d34-45c0-ada2-662aa0757ded 2026-02-19T16:26:17.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e4e3f6dc-8d34-45c0-ada2-662aa0757ded' 2026-02-19T16:26:17.028 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 e4e3f6dc-8d34-45c0-ada2-662aa0757ded 2026-02-19T16:26:17.028 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:26:17.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCpOZdpWPc6AhAAJRi6O/pnPDtZlUIpukqrNA== 2026-02-19T16:26:17.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCpOZdpWPc6AhAAJRi6O/pnPDtZlUIpukqrNA=="}' 2026-02-19T16:26:17.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e4e3f6dc-8d34-45c0-ada2-662aa0757ded -i td/test-erasure-eio/3/new.json 2026-02-19T16:26:17.277 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:26:17.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:26:17.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCpOZdpWPc6AhAAJRi6O/pnPDtZlUIpukqrNA== --osd-uuid e4e3f6dc-8d34-45c0-ada2-662aa0757ded 2026-02-19T16:26:17.308 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.305+0000 7fe77e224780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:17.309 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.306+0000 7fe77e224780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:17.310 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.307+0000 7fe77e224780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:17.310 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.307+0000 7fe77e224780 -1 bdev(0x4302c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:26:17.310 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.308+0000 7fe77e224780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:26:17.638 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:26:17.638 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:26:17.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:26:17.639 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:26:17.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:26:17.978 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:26:17.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:26:17.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:26:17.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:26:17.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:26:17.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:26:17.998 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.995+0000 7fd5d720b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:17.999 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.997+0000 7fd5d720b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:18.000 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:17.998+0000 7fd5d720b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:18.130 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:18.127+0000 7fd5d720b780 -1 Falling back to public interface 2026-02-19T16:26:18.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:26:18.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:26:18.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:26:18.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:26:18.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:26:18.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:18.208 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:26:18.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:26:18.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:18.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:26:18.441 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:18.438+0000 7fd5d720b780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:26:18.448 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:26:19.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:26:19.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:19.449 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:26:19.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:26:19.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:19.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:26:19.685 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2553554643,v1:127.0.0.1:6827/2553554643] [v2:127.0.0.1:6828/2553554643,v1:127.0.0.1:6829/2553554643] exists,up e4e3f6dc-8d34-45c0-ada2-662aa0757ded 2026-02-19T16:26:19.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:26:19.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:26:19.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:19.691 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:26:19.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:26:19.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T16:26:19.693 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:26:19.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5f9858ff-fbd9-44ea-beb6-728f5a99cbf2 2026-02-19T16:26:19.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 5f9858ff-fbd9-44ea-beb6-728f5a99cbf2' 2026-02-19T16:26:19.693 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 5f9858ff-fbd9-44ea-beb6-728f5a99cbf2 2026-02-19T16:26:19.693 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:26:19.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCrOZdpuc63KRAAsYEhmQBMOQ7FLXYRaf31Jw== 2026-02-19T16:26:19.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCrOZdpuc63KRAAsYEhmQBMOQ7FLXYRaf31Jw=="}' 2026-02-19T16:26:19.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5f9858ff-fbd9-44ea-beb6-728f5a99cbf2 -i td/test-erasure-eio/4/new.json 2026-02-19T16:26:19.935 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:26:19.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T16:26:19.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCrOZdpuc63KRAAsYEhmQBMOQ7FLXYRaf31Jw== --osd-uuid 5f9858ff-fbd9-44ea-beb6-728f5a99cbf2 2026-02-19T16:26:19.964 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:19.961+0000 7f7ea31dc780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:19.966 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:19.963+0000 7f7ea31dc780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:19.966 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:19.964+0000 7f7ea31dc780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:19.967 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:19.964+0000 7f7ea31dc780 -1 bdev(0x2e04c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:26:19.967 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:19.964+0000 7f7ea31dc780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T16:26:20.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T16:26:20.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:26:20.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:26:20.310 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:26:20.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:26:20.638 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:26:20.639 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:26:20.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:26:20.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:26:20.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:26:20.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:26:20.657 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:20.654+0000 7fb54382f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:20.658 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:20.655+0000 7fb54382f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:20.659 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:20.656+0000 7fb54382f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:20.797 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:20.794+0000 7fb54382f780 -1 Falling back to public interface 2026-02-19T16:26:20.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:26:20.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:26:20.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:26:20.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:26:20.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:26:20.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:20.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:26:20.872 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:26:20.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:20.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:26:21.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:26:21.117 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:21.114+0000 7fb54382f780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:26:22.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:26:22.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:22.107 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:26:22.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:26:22.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:22.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:26:22.339 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 36 up_thru 37 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/406358337,v1:127.0.0.1:6835/406358337] [v2:127.0.0.1:6836/406358337,v1:127.0.0.1:6837/406358337] exists,up 5f9858ff-fbd9-44ea-beb6-728f5a99cbf2 2026-02-19T16:26:22.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:26:22.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:26:22.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:26:22.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:26:22.386 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:26:22.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:26:10.526+0000 7fb1f5410780 0 load: jerasure load: lrc 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:26:22.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T16:26:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:26:22.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:26:23.057 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:26:23.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:26:24.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:26:24.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:26:24.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:26:24.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:26:24.068 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:26:24.068 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:26:24.069 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:26:24.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:26:24.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:26:24.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:26:24.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:26:24.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:26:24.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:26:24.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:26:24.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:26:24.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:26:24.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:26:24.380 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:26:24.381 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:26:24.381 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:26:24.381 INFO:tasks.workunit.client.0.trial127.stderr:4' 2026-02-19T16:26:24.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:26:24.381 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:24.381 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:26:24.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T16:26:24.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T16:26:24.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T16:26:24.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:24.447 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:26:24.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-19T16:26:24.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-19T16:26:24.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558' 2026-02-19T16:26:24.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:24.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:26:24.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T16:26:24.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T16:26:24.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411333' 2026-02-19T16:26:24.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:24.579 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:26:24.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116996 2026-02-19T16:26:24.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116996 2026-02-19T16:26:24.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411333 3-115964116996' 2026-02-19T16:26:24.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:24.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:26:24.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822660 2026-02-19T16:26:24.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822660 2026-02-19T16:26:24.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411333 3-115964116996 4-154618822660' 2026-02-19T16:26:24.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:24.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-19T16:26:24.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:24.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:26:24.706 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T16:26:24.706 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:24.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-19T16:26:24.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T16:26:24.708 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803781 2026-02-19T16:26:24.708 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:26:24.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T16:26:24.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:24.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-19T16:26:24.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:24.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:26:24.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-19T16:26:24.934 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:24.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-19T16:26:24.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-19T16:26:24.935 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607558 2026-02-19T16:26:24.935 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:26:25.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-19T16:26:25.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:25.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-19T16:26:25.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:25.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:26:25.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T16:26:25.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:25.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-19T16:26:25.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T16:26:25.168 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411333 2026-02-19T16:26:25.168 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:26:25.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411331 -lt 77309411333 2026-02-19T16:26:25.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:26:26.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:26:26.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:26:26.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T16:26:26.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:26.635 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116996 2026-02-19T16:26:26.635 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:26.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:26:26.636 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116996 2026-02-19T16:26:26.636 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:26.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116996 2026-02-19T16:26:26.637 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116996' 2026-02-19T16:26:26.638 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 115964116996 2026-02-19T16:26:26.638 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:26:26.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116996 -lt 115964116996 2026-02-19T16:26:26.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:26.866 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822660 2026-02-19T16:26:26.866 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:26.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:26:26.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822660 2026-02-19T16:26:26.868 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:26.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822660 2026-02-19T16:26:26.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822660' 2026-02-19T16:26:26.869 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 154618822660 2026-02-19T16:26:26.869 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:26:27.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-19T16:26:27.100 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:26:27.101 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:27.101 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:27.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:26:27.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:26:27.423 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:26:27.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:26:27.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:26:27.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:26:27.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:26:27.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:26:27.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:26:27.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:26:27.649 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:27.649 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:27.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:26:27.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:26:27.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:26:27.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:24.224602+0000 0'0 42:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T16:26:22.990889+0000 0'0 2026-02-19T16:26:22.990889+0000 0 0 periodic scrub scheduled @ 2026-02-20T20:13:11.174054+0000 0 0 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218948+0000 0'0 42:19 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-20T21:39:29.799879+0000 0 0 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218974+0000 0'0 42:103 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-20T20:18:34.293584+0000 0 0 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218900+0000 0'0 42:19 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-21T03:52:56.740996+0000 0 0 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:16.983878+0000 0'0 43:69 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-21T03:49:58.519352+0000 0 0 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-19T16:26:28.199 INFO:tasks.workunit.client.0.trial127.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T16:26:28.200 INFO:tasks.workunit.client.0.trial127.stderr:dumped pgs 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:26:28.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:26:28.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:26:28.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:26:28.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:26:28.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:26:28.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:28.226 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-19T16:26:28.226 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:26:28.227 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:26:28.227 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:26:28.227 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:26:28.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:26:28.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:26:28.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:26:28.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:26:28.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:26:28.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:26:28.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:26:28.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-19T16:26:28.985 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already down. 2026-02-19T16:26:28.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-19T16:26:29.239 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already out. 2026-02-19T16:26:29.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-19T16:26:29.465 INFO:tasks.workunit.client.0.trial127.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T16:26:29.465 INFO:tasks.workunit.client.0.trial127.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:24.224602+0000 0'0 42:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T16:26:22.990889+0000 0'0 2026-02-19T16:26:22.990889+0000 0 0 periodic scrub scheduled @ 2026-02-20T20:13:11.174054+0000 0 0 2026-02-19T16:26:29.465 INFO:tasks.workunit.client.0.trial127.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218948+0000 0'0 43:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-20T21:39:29.799879+0000 0 0 2026-02-19T16:26:29.465 INFO:tasks.workunit.client.0.trial127.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218974+0000 0'0 43:105 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-20T20:18:34.293584+0000 0 0 2026-02-19T16:26:29.465 INFO:tasks.workunit.client.0.trial127.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218900+0000 0'0 43:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-21T03:52:56.740996+0000 0 0 2026-02-19T16:26:29.466 INFO:tasks.workunit.client.0.trial127.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:16.983878+0000 0'0 43:69 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-21T03:49:58.519352+0000 0 0 2026-02-19T16:26:29.466 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-19T16:26:29.466 INFO:tasks.workunit.client.0.trial127.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T16:26:29.466 INFO:tasks.workunit.client.0.trial127.stderr:dumped pgs 2026-02-19T16:26:29.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-19T16:26:29.474 INFO:tasks.workunit.client.0.trial127.stderr:4+0 records in 2026-02-19T16:26:29.474 INFO:tasks.workunit.client.0.trial127.stderr:4+0 records out 2026-02-19T16:26:29.474 INFO:tasks.workunit.client.0.trial127.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.6674e-05 s, 61.4 MB/s 2026-02-19T16:26:29.475 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-19T16:26:29.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.633 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.634 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.652 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.771 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.771 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.788 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:29.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:29.992 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.113 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.165 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.199 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.251 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.285 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.302 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.303 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.319 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.320 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.337 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.573 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.643 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.747 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.747 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.798 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.866 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.866 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.982 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.982 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:30.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:30.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.101 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.101 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:31.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-19T16:26:31.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-19T16:26:31.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:26:31.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:26:31.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:26:31.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-19T16:26:31.189 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:26:31.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:26:31.421 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:26:31.421 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:26:31.421 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:26:31.421 INFO:tasks.workunit.client.0.trial127.stderr:2147483647' 2026-02-19T16:26:31.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T16:26:31.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T16:26:31.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:26:31.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-19T16:26:31.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:26:31.422 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T16:26:31.424 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:26:31.425 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:26:31.425 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:31.425 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:31.425 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:31.425 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:26:31.425 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:26:31.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:26:31.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:26:31.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:26:31.475 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:26:31.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-19T16:26:31.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-19T16:26:31.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:26:31.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:26:31.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:26:31.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-19T16:26:31.525 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-19T16:26:31.526 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:26:31.755 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:2147483647' 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-19T16:26:31.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:26:31.757 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T16:26:31.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:26:31.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:31.759 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:26:31.760 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:26:31.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:26:31.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:26:31.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:26:31.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-19T16:26:31.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:26:31.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:26:31.807 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:26:31.807 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:31.807 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:31.807 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:26:31.807 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:26:31.807 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:26:31.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:26:31.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:26:31.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:31.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:26:31.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:26:31.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:26:31.859 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:26:31.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:26:31.860 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:26:31.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:26:31.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:26:31.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:26:31.860 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:26:31.872 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:31.871+0000 7f5af320e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:31.874 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:31.872+0000 7f5af320e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:31.875 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:31.873+0000 7f5af320e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:26:31.994 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:31.992+0000 7f5af320e780 -1 Falling back to public interface 2026-02-19T16:26:32.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:26:32.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:26:32.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:26:32.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:26:32.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:26:32.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:32.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:26:32.085 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:26:32.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:32.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:26:32.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:26:32.347 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:26:32.345+0000 7f5af320e780 -1 osd.2 43 log_to_monitors true 2026-02-19T16:26:33.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:26:33.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:26:33.313 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:26:33.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:26:33.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:26:33.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:26:33.538 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up out weight 0 up_from 49 up_thru 20 down_at 44 last_clean_interval [18,43) [v2:127.0.0.1:6818/1545671854,v1:127.0.0.1:6819/1545671854] [v2:127.0.0.1:6820/1545671854,v1:127.0.0.1:6821/1545671854] exists,up 2b1aec2a-b43a-4f83-bddc-edb82f528dc1 2026-02-19T16:26:33.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:26:33.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:26:33.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:26:33.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-19T16:26:33.813 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already in. 2026-02-19T16:26:33.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-19T16:26:48.823 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-19T16:26:48.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-19T16:26:48.825 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-19T16:26:48.825 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-19T16:26:48.825 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-19T16:26:48.826 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-19T16:26:48.826 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-19T16:26:49.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-19T16:26:49.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-19T16:26:49.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-19T16:26:49.054 INFO:tasks.workunit.client.0.trial127.stdout:active+recovery_unfound+degraded 2026-02-19T16:26:49.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-19T16:26:49.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-19T16:26:49.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-19T16:26:49.273 INFO:tasks.workunit.client.0.trial127.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T16:26:49.273 INFO:tasks.workunit.client.0.trial127.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-19T16:26:33.850234+0000 47'101 53:358 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T16:26:22.990889+0000 0'0 2026-02-19T16:26:22.990889+0000 0 0 no scrub is scheduled 0 0 2026-02-19T16:26:49.274 INFO:tasks.workunit.client.0.trial127.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:33.848546+0000 0'0 53:58 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-21T03:40:27.309676+0000 0 0 2026-02-19T16:26:49.274 INFO:tasks.workunit.client.0.trial127.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218974+0000 0'0 53:123 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-20T20:18:34.293584+0000 0 0 2026-02-19T16:26:49.274 INFO:tasks.workunit.client.0.trial127.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:22.218900+0000 0'0 53:40 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-21T03:52:56.740996+0000 0 0 2026-02-19T16:26:49.274 INFO:tasks.workunit.client.0.trial127.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T16:26:33.848605+0000 0'0 53:107 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T16:26:08.600541+0000 0'0 2026-02-19T16:26:08.600541+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:55:28.681666+0000 0 0 2026-02-19T16:26:49.274 INFO:tasks.workunit.client.0.trial127.stdout: 2026-02-19T16:26:49.274 INFO:tasks.workunit.client.0.trial127.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T16:26:49.274 INFO:tasks.workunit.client.0.trial127.stderr:dumped pgs 2026-02-19T16:26:49.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:26:49.343 INFO:tasks.workunit.client.0.trial127.stdout:{ 2026-02-19T16:26:49.343 INFO:tasks.workunit.client.0.trial127.stdout: "num_missing": 1, 2026-02-19T16:26:49.343 INFO:tasks.workunit.client.0.trial127.stdout: "num_unfound": 1, 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "objects": [ 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "oid": { 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "oid": "obj75", 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "key": "", 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "snapid": -2, 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "hash": 3565005477, 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "max": 0, 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "pool": 2, 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "namespace": "" 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.344 INFO:tasks.workunit.client.0.trial127.stdout: "need": "47'76", 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "have": "0'0", 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "flags": "none", 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "locations": [ 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: ] 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "state": "NotRecovering", 2026-02-19T16:26:49.345 INFO:tasks.workunit.client.0.trial127.stdout: "available_might_have_unfound": true, 2026-02-19T16:26:49.346 INFO:tasks.workunit.client.0.trial127.stdout: "might_have_unfound": [], 2026-02-19T16:26:49.346 INFO:tasks.workunit.client.0.trial127.stdout: "more": false 2026-02-19T16:26:49.346 INFO:tasks.workunit.client.0.trial127.stdout:} 2026-02-19T16:26:49.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-19T16:26:49.410 INFO:tasks.workunit.client.0.trial127.stdout:{ 2026-02-19T16:26:49.410 INFO:tasks.workunit.client.0.trial127.stdout: "snap_trimq": "[]", 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: "snap_trimq_len": 0, 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+recovery_unfound+degraded", 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: "epoch": 53, 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:26:49.411 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: "acting_recovery_backfill": [ 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: "2(4)", 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:26:49.412 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "info": { 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s0", 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "0", 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'101", 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'75", 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "0'0", 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 101, 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "52", 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:26:49.413 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 52, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 52, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 52, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.414 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'101", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 359, 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 53, 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+recovery_unfound+degraded", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:26:33.850234+0000", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:26:33.850234+0000", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:26:33.850234+0000", 2026-02-19T16:26:49.415 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:26:33.850234+0000", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:26:28.223313+0000", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:26:33.848656+0000", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:26:33.848656+0000", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:26:33.850234+0000", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:26:33.847361+0000", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:26:33.850234+0000", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 52, 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "0'0", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "0'0", 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:26:49.416 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 0, 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:26:49.417 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:26:49.418 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 413696, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 101, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 505, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 1, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 1, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 3, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 1, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 101, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:26:49.419 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 101, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 404, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:26:49.420 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:26:49.421 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.422 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 100 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.423 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),4(2)", 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 1 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:26:49.424 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "peer_info": [ 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "peer": "0(3)", 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s3", 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "3", 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'101", 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'101", 2026-02-19T16:26:49.425 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "0'0", 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 101, 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.426 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 52, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 52, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 52, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.427 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'101", 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 347, 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 52, 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+remapped", 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:26:28.223313+0000", 2026-02-19T16:26:49.428 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:26:28.635364+0000", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:26:28.635217+0000", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 52, 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "0'0", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "0'0", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:26:49.429 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 0, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:26:49.430 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-21T03:52:57.948293+0000", 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 413696, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 101, 2026-02-19T16:26:49.431 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 505, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 101, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 101, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 101, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 404, 2026-02-19T16:26:49.432 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:26:49.433 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.434 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.435 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 101 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.436 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.437 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "peer": "1(1)", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s1", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "1", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'101", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'101", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "0'0", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 101, 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:26:49.438 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 52, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 52, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 52, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.439 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'101", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 131, 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 47, 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+remapped", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.440 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:26:28.223313+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:26:28.635364+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:26:28.635217+0000", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 52, 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "0'0", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "0'0", 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.441 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 0, 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:26:49.442 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-21T03:52:57.948293+0000", 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 413696, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 101, 2026-02-19T16:26:49.443 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 505, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 1, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 101, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 0, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 101, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 101, 2026-02-19T16:26:49.444 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 404, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:26:49.445 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:26:49.446 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:26:49.447 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 101 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:26:49.448 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "peer": "2(4)", 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s4", 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "4", 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'101", 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'75", 2026-02-19T16:26:49.449 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "0'0", 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 101, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 52, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 52, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 52, 2026-02-19T16:26:49.450 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'101", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 137, 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 51, 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+remapped", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:26:33.645153+0000", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:26:33.545988+0000", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:26:33.545988+0000", 2026-02-19T16:26:49.451 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:26:33.492822+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:26:28.223313+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:26:33.645153+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:26:33.645153+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:26:33.645153+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 52, 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "0'0", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "0'0", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.452 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 0, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:26:49.453 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 413696, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 101, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 505, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 1, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 101, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 101, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 404, 2026-02-19T16:26:49.454 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:26:49.455 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.456 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [], 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [], 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:26:49.457 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "peer": "4(2)", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s2", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "shard": "2", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "last_update": "47'101", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "last_complete": "47'101", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "log_tail": "0'0", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "last_user_version": 101, 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill": "MAX", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete": [], 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [], 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "history": { 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_created": 41, 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "epoch_pool_created": 41, 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_started": 52, 2026-02-19T16:26:49.458 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_interval_clean": 41, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_split": 0, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_marked_full": 0, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "same_up_since": 52, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "same_interval_since": 52, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "same_primary_since": 41, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "prior_readable_until_ub": 0 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "stats": { 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "version": "47'101", 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "reported_seq": 131, 2026-02-19T16:26:49.459 INFO:tasks.workunit.client.0.trial127.stdout: "reported_epoch": 47, 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "state": "active+remapped", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_fresh": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_change": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_active": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_peered": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean": "2026-02-19T16:26:28.223313+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_active": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_became_peered": "2026-02-19T16:26:28.636553+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_unstale": "2026-02-19T16:26:31.184822+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_undegraded": "2026-02-19T16:26:28.635364+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_fullsized": "2026-02-19T16:26:28.635217+0000", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "mapping_epoch": 52, 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "log_start": "0'0", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_start": "0'0", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "created": 41, 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_clean": 42, 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "parent": "0.0", 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "parent_split_bits": 0, 2026-02-19T16:26:49.460 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub": "0'0", 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub": "0'0", 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "last_deep_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "last_clean_scrub_stamp": "2026-02-19T16:26:22.990889+0000", 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "objects_scrubbed": 0, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "log_size": 101, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "log_dups_size": 0, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "ondisk_log_size": 101, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "stats_invalid": false, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "dirty_stats_invalid": false, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "omap_stats_invalid": false, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_stats_invalid": false, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "pin_stats_invalid": false, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "manifest_stats_invalid": false, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrimq_len": 0, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "last_scrub_duration": 0, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-21T03:52:57.948293+0000", 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "scrub_duration": 0, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "objects_trimmed": 0, 2026-02-19T16:26:49.461 INFO:tasks.workunit.client.0.trial127.stdout: "snaptrim_duration": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "stat_sum": { 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes": 413696, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects": 101, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_clones": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_object_copies": 505, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_missing": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_degraded": 101, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_misplaced": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_unfound": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_dirty": 101, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_whiteouts": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_read": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_read_kb": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_write": 101, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_write_kb": 404, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_scrub_errors": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_deep_scrub_errors": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_recovered": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_recovered": 0, 2026-02-19T16:26:49.462 INFO:tasks.workunit.client.0.trial127.stdout: "num_keys_recovered": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_omap": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_kb": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_kb": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_promote": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_high": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_flush_mode_low": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_some": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_evict_mode_full": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_pinned": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_legacy_snapsets": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_large_omap_objects": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_manifest": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_bytes": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_omap_keys": 0, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "num_objects_repaired": 0 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.463 INFO:tasks.workunit.client.0.trial127.stdout: "up": [ 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "acting": [ 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 3, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 1, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 4, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 0, 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: 2 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "avail_no_missing": [ 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "3(0)", 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "0(3)", 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "1(1)", 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "4(2)" 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "object_location_counts": [ 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: "objects": 101 2026-02-19T16:26:49.464 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "blocked_by": [], 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "up_primary": 3, 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "acting_primary": 3, 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "purged_snaps": [] 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "empty": 0, 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "dne": 0, 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "incomplete": 0, 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "last_epoch_started": 53, 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "hit_set_history": { 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "current_last_update": "0'0", 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "history": [] 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "recovery_state": [ 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "name": "Started/Primary/Active", 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "enter_time": "2026-02-19T16:26:33.847213+0000", 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "might_have_unfound": [ 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.465 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "0(3)", 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "status": "already probed" 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "1(1)", 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "status": "already probed" 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "2(4)", 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "status": "already probed" 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "osd": "4(2)", 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "status": "already probed" 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "recovery_progress": { 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "backfill_targets": [], 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "waiting_on_backfill": [], 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "last_backfill_started": "MIN", 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "backfill_info": { 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "begin": "MIN", 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "end": "MIN", 2026-02-19T16:26:49.466 INFO:tasks.workunit.client.0.trial127.stdout: "objects": [] 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "peer_backfill_info": [], 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "backfills_in_flight": [], 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "recovering": [], 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "pg_backend": { 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "recovery_ops": [], 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "read_ops": [] 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: { 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "name": "Started", 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "enter_time": "2026-02-19T16:26:33.746058+0000" 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: ], 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "scrubber": { 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "active": false, 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "must_scrub": false, 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "must_deep_scrub": false, 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "must_repair": false, 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "sched_time": "2026-02-21T03:52:57.948293+0000", 2026-02-19T16:26:49.467 INFO:tasks.workunit.client.0.trial127.stdout: "schedule": "not registered for scrubbing", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "urgency": "periodic-regular", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "shallow-target": { 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s0", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "level": "shallow", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "urgency": "periodic-regular", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "sched_time": "2026-02-21T03:52:57.948293+0000", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "orig_sched_time": "2026-02-21T03:52:57.948293+0000", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "last_issue": "ok", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "forced": false, 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "eligible": false, 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "queued": false, 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "active": false 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "deep-target": { 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "pgid": "2.0s0", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "level": "deep", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "urgency": "periodic-regular", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "sched_time": "2026-02-27T18:05:28.344694+0000", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "orig_sched_time": "2026-02-27T18:05:28.344694+0000", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "last_issue": "ok", 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "forced": false, 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "eligible": false, 2026-02-19T16:26:49.468 INFO:tasks.workunit.client.0.trial127.stdout: "queued": false, 2026-02-19T16:26:49.469 INFO:tasks.workunit.client.0.trial127.stdout: "active": false 2026-02-19T16:26:49.469 INFO:tasks.workunit.client.0.trial127.stdout: } 2026-02-19T16:26:49.469 INFO:tasks.workunit.client.0.trial127.stdout: }, 2026-02-19T16:26:49.469 INFO:tasks.workunit.client.0.trial127.stdout: "agent_state": {} 2026-02-19T16:26:49.469 INFO:tasks.workunit.client.0.trial127.stdout:} 2026-02-19T16:26:49.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:26:49.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-19T16:26:49.481 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:26:49.481 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-19T16:26:49.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-19T16:26:49.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-19T16:26:49.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T16:26:49.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-19T16:26:49.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-19T16:26:49.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-19T16:26:49.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-19T16:26:54.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-19T16:26:54.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-19T16:26:54.683 INFO:tasks.workunit.client.0.trial127.stderr:pg has no unfound objects 2026-02-19T16:26:54.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-19T16:26:54.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:26:54.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:26:54.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:26:54.689 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:26:54.690 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:26:54.690 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:26:54.690 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:26:54.690 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:26:54.690 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:26:54.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:26:54.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:26:54.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:26:54.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:26:54.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:26:54.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:26:55.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:26:55.004 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:26:55.004 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:26:55.004 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:26:55.004 INFO:tasks.workunit.client.0.trial127.stderr:4' 2026-02-19T16:26:55.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:26:55.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:55.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:26:55.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803789 2026-02-19T16:26:55.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803789 2026-02-19T16:26:55.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789' 2026-02-19T16:26:55.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:55.072 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:26:55.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607566 2026-02-19T16:26:55.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607566 2026-02-19T16:26:55.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566' 2026-02-19T16:26:55.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:55.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:26:55.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397511 2026-02-19T16:26:55.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397511 2026-02-19T16:26:55.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566 2-210453397511' 2026-02-19T16:26:55.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:55.202 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:26:55.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117004 2026-02-19T16:26:55.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117004 2026-02-19T16:26:55.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566 2-210453397511 3-115964117004' 2026-02-19T16:26:55.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:26:55.266 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:26:55.329 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822668 2026-02-19T16:26:55.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822668 2026-02-19T16:26:55.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566 2-210453397511 3-115964117004 4-154618822668' 2026-02-19T16:26:55.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:55.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803789 2026-02-19T16:26:55.330 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:55.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:26:55.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803789 2026-02-19T16:26:55.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:55.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803789 2026-02-19T16:26:55.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-19T16:26:55.333 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803789 2026-02-19T16:26:55.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:26:55.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-19T16:26:55.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:26:56.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:26:56.564 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:26:56.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-19T16:26:56.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:56.792 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607566 2026-02-19T16:26:56.792 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:56.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:26:56.793 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607566 2026-02-19T16:26:56.793 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:56.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607566 2026-02-19T16:26:56.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607566' 2026-02-19T16:26:56.795 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607566 2026-02-19T16:26:56.795 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:26:57.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607566 -lt 51539607566 2026-02-19T16:26:57.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:57.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-210453397511 2026-02-19T16:26:57.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:57.026 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:26:57.026 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-210453397511 2026-02-19T16:26:57.026 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:57.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397511 2026-02-19T16:26:57.028 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 210453397511 2026-02-19T16:26:57.028 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 210453397511' 2026-02-19T16:26:57.028 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:26:57.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397511 -lt 210453397511 2026-02-19T16:26:57.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:57.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117004 2026-02-19T16:26:57.257 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:57.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:26:57.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117004 2026-02-19T16:26:57.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:57.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117004 2026-02-19T16:26:57.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117004' 2026-02-19T16:26:57.259 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 115964117004 2026-02-19T16:26:57.260 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:26:57.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117004 -lt 115964117004 2026-02-19T16:26:57.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:26:57.490 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822668 2026-02-19T16:26:57.490 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:26:57.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:26:57.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822668 2026-02-19T16:26:57.492 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:26:57.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822668 2026-02-19T16:26:57.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822668' 2026-02-19T16:26:57.493 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 154618822668 2026-02-19T16:26:57.493 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:26:57.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822668 -lt 154618822668 2026-02-19T16:26:57.720 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:26:57.720 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:57.720 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:58.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:26:58.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:26:58.050 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:26:58.050 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:26:58.050 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:26:58.050 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:26:58.050 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:26:58.050 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:26:58.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:26:58.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:26:58.273 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:26:58.273 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:26:58.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:26:58.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:26:58.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:26:58.592 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-19T16:26:58.592 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.592 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-19T16:26:58.592 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-19T16:26:58.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-19T16:26:58.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.644 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-19T16:26:58.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-19T16:26:58.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-19T16:26:58.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-19T16:26:58.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-19T16:26:58.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-19T16:26:58.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.744 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-19T16:26:58.745 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-19T16:26:58.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-19T16:26:58.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-19T16:26:58.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-19T16:26:58.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-19T16:26:58.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-19T16:26:58.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-19T16:26:58.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.878 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-19T16:26:58.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-19T16:26:58.896 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-19T16:26:58.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-19T16:26:58.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-19T16:26:58.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-19T16:26:58.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-19T16:26:58.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:58.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:58.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-19T16:26:58.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-19T16:26:59.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-19T16:26:59.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-19T16:26:59.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.052 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-19T16:26:59.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.071 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-19T16:26:59.071 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-19T16:26:59.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-19T16:26:59.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-19T16:26:59.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-19T16:26:59.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-19T16:26:59.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-19T16:26:59.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-19T16:26:59.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-19T16:26:59.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-19T16:26:59.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-19T16:26:59.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-19T16:26:59.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-19T16:26:59.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-19T16:26:59.294 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-19T16:26:59.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.326 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-19T16:26:59.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-19T16:26:59.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-19T16:26:59.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-19T16:26:59.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-19T16:26:59.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-19T16:26:59.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-19T16:26:59.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-19T16:26:59.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.460 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-19T16:26:59.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-19T16:26:59.478 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-19T16:26:59.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-19T16:26:59.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-19T16:26:59.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-19T16:26:59.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-19T16:26:59.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-19T16:26:59.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-19T16:26:59.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-19T16:26:59.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-19T16:26:59.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-19T16:26:59.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-19T16:26:59.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-19T16:26:59.683 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-19T16:26:59.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-19T16:26:59.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-19T16:26:59.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-19T16:26:59.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-19T16:26:59.769 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-19T16:26:59.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-19T16:26:59.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.818 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-19T16:26:59.819 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-19T16:26:59.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-19T16:26:59.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.869 INFO:tasks.workunit.client.0.trial127.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-19T16:26:59.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-19T16:26:59.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.886 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-19T16:26:59.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-19T16:26:59.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-19T16:26:59.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-19T16:26:59.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-19T16:26:59.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.971 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-19T16:26:59.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-19T16:26:59.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:26:59.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:26:59.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-19T16:26:59.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-19T16:27:00.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-19T16:27:00.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-19T16:27:00.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-19T16:27:00.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-19T16:27:00.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.091 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-19T16:27:00.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-19T16:27:00.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-19T16:27:00.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-19T16:27:00.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-19T16:27:00.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-19T16:27:00.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-19T16:27:00.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-19T16:27:00.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-19T16:27:00.229 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-19T16:27:00.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.262 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-19T16:27:00.263 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T16:27:00.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-19T16:27:00.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-19T16:27:00.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T16:27:00.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:27:00.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:27:00.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:27:00.568 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:27:00.576 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:27:00.855 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:27:00.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:27:00.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:27:00.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:27:00.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:27:00.864 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:27:00.865 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:27:00.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:27:00.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:27:00.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:27:00.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:27:00.974 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:27:00.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:27:00.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:27:00.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:27:00.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:27:00.976 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:27:00.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:00.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:27:00.977 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:27:00.978 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:00.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:27:00.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:27:00.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:27:01.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:27:01.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:01.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:01.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:27:01.002 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:27:01.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:27:01.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:27:01.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:27:01.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:27:01.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:27:01.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:27:01.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:27:01.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:27:01.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:27:01.006 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:27:01.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:01.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:27:01.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:27:01.008 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:01.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:27:01.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:27:01.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:27:01.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:27:01.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:01.011 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:01.011 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:27:01.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:27:01.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:27:01.012 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:27:01.013 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:27:01.013 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:01.013 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:01.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:27:01.014 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:27:01.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:27:01.034 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:27:01.034 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:01.034 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:01.034 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:01.034 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:01.034 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:01.034 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:01.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:27:01.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:27:01.060 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:27:01.061 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:01.061 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:01.061 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:01.061 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:27:01.061 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:27:01.061 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:27:01.103 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:27:01.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:27:01.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:27:01.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:27:01.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:27:01.104 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:27:01.104 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:27:01.105 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:27:01.105 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:01.105 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:01.105 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:01.105 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:27:01.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:27:01.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:27:01.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:27:01.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:27:01.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:27:01.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:27:01.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:27:01.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:27:01.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:27:01.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:27:01.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:01.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:01.244 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:01.244 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:01.244 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:01.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:01.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:27:01.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:27:01.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:27:01.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:27:01.356 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:27:01.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:27:02.365 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:27:02.365 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:27:02.366 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:27:02.366 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:02.366 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:02.366 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:02.366 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:27:02.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:27:02.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:27:02.405 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:27:02.409 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:27:02.410 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:27:01.050+0000 7fca2dc38c00 0 load: jerasure load: lrc 2026-02-19T16:27:02.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-19T16:27:02.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-19T16:27:02.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-19T16:27:02.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-19T16:27:02.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T16:27:02.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:27:02.412 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T16:27:02.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T16:27:02.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:02.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:27:02.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:27:02.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:02.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:02.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:02.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:27:02.420 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:02.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a34051f1-2560-40dd-9816-aaad00e9bfd2 2026-02-19T16:27:02.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a34051f1-2560-40dd-9816-aaad00e9bfd2' 2026-02-19T16:27:02.420 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 a34051f1-2560-40dd-9816-aaad00e9bfd2 2026-02-19T16:27:02.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:02.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDWOZdpdYZpGRAAAJDeRS9L4082vvCWwP6OVg== 2026-02-19T16:27:02.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDWOZdpdYZpGRAAAJDeRS9L4082vvCWwP6OVg=="}' 2026-02-19T16:27:02.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a34051f1-2560-40dd-9816-aaad00e9bfd2 -i td/test-erasure-eio/0/new.json 2026-02-19T16:27:02.532 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:02.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:27:02.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDWOZdpdYZpGRAAAJDeRS9L4082vvCWwP6OVg== --osd-uuid a34051f1-2560-40dd-9816-aaad00e9bfd2 2026-02-19T16:27:02.556 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:02.554+0000 7f796e02d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:02.557 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:02.555+0000 7f796e02d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:02.558 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:02.556+0000 7f796e02d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:02.559 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:02.556+0000 7f796e02d780 -1 bdev(0x423ac00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:02.559 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:02.557+0000 7f796e02d780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:27:02.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:27:02.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:02.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:27:02.988 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:27:02.989 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:03.092 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:27:03.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:27:03.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:03.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:03.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:03.094 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:03.108 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:03.107+0000 7f3bc320e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:03.110 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:03.108+0000 7f3bc320e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:03.110 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:03.109+0000 7f3bc320e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:03.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:27:03.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:03.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:27:03.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:03.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:03.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:03.193 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:03.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:03.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:03.194 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:27:03.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:03.294 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:03.292+0000 7f3bc320e780 -1 Falling back to public interface 2026-02-19T16:27:03.569 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:03.567+0000 7f3bc320e780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:27:04.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:04.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:04.289 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:04.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:04.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:04.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:27:04.525 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/599540229,v1:127.0.0.1:6801/599540229] [v2:127.0.0.1:6802/599540229,v1:127.0.0.1:6803/599540229] exists,up a34051f1-2560-40dd-9816-aaad00e9bfd2 2026-02-19T16:27:04.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:04.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:04.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:04.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:04.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:04.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:27:04.533 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:04.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=daf8f740-8f7d-4df6-bf49-2e5530a98c55 2026-02-19T16:27:04.533 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 daf8f740-8f7d-4df6-bf49-2e5530a98c55' 2026-02-19T16:27:04.533 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 daf8f740-8f7d-4df6-bf49-2e5530a98c55 2026-02-19T16:27:04.533 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:04.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDYOZdp1EwIIBAADGBccsJkFQP5VIGKMIp+Jg== 2026-02-19T16:27:04.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDYOZdp1EwIIBAADGBccsJkFQP5VIGKMIp+Jg=="}' 2026-02-19T16:27:04.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new daf8f740-8f7d-4df6-bf49-2e5530a98c55 -i td/test-erasure-eio/1/new.json 2026-02-19T16:27:04.769 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:04.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:27:04.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDYOZdp1EwIIBAADGBccsJkFQP5VIGKMIp+Jg== --osd-uuid daf8f740-8f7d-4df6-bf49-2e5530a98c55 2026-02-19T16:27:04.794 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:04.792+0000 7fb96f413780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:04.795 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:04.793+0000 7fb96f413780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:04.796 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:04.794+0000 7fb96f413780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:04.796 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:04.794+0000 7fb96f413780 -1 bdev(0x35fec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:04.797 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:04.794+0000 7fb96f413780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:27:05.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:27:05.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:05.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:27:05.126 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:27:05.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:05.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:27:05.446 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:27:05.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:05.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:05.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:05.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:05.460 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:05.459+0000 7f2aaf02f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:05.462 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:05.460+0000 7f2aaf02f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:05.463 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:05.461+0000 7f2aaf02f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:05.636 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:05.634+0000 7f2aaf02f780 -1 Falling back to public interface 2026-02-19T16:27:05.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:27:05.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:05.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:27:05.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:05.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:05.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:05.678 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:05.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:05.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:05.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:27:05.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:05.967 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:05.965+0000 7f2aaf02f780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:27:06.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:06.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:06.907 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:06.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:06.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:06.907 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:27:07.137 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/386437201,v1:127.0.0.1:6811/386437201] [v2:127.0.0.1:6812/386437201,v1:127.0.0.1:6813/386437201] exists,up daf8f740-8f7d-4df6-bf49-2e5530a98c55 2026-02-19T16:27:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:27:07.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:07.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:07.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c552d47c-f028-42dd-aba4-9523cc4f84a1 2026-02-19T16:27:07.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c552d47c-f028-42dd-aba4-9523cc4f84a1' 2026-02-19T16:27:07.145 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 c552d47c-f028-42dd-aba4-9523cc4f84a1 2026-02-19T16:27:07.145 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:07.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDbOZdpNF/kCBAA9dZ27wY7xDAdMnzaTzRI3g== 2026-02-19T16:27:07.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDbOZdpNF/kCBAA9dZ27wY7xDAdMnzaTzRI3g=="}' 2026-02-19T16:27:07.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c552d47c-f028-42dd-aba4-9523cc4f84a1 -i td/test-erasure-eio/2/new.json 2026-02-19T16:27:07.381 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:27:07.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:27:07.389 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDbOZdpNF/kCBAA9dZ27wY7xDAdMnzaTzRI3g== --osd-uuid c552d47c-f028-42dd-aba4-9523cc4f84a1 2026-02-19T16:27:07.404 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:07.403+0000 7fb3daa24780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:07.406 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:07.404+0000 7fb3daa24780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:07.407 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:07.405+0000 7fb3daa24780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:07.407 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:07.406+0000 7fb3daa24780 -1 bdev(0x4004c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:07.407 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:07.406+0000 7fb3daa24780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:27:07.723 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:27:07.723 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:07.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:27:07.724 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:27:07.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:08.041 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:27:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:27:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:08.056 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:08.054+0000 7f53c2655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:08.057 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:08.055+0000 7f53c2655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:08.058 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:08.056+0000 7f53c2655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:08.243 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:08.242+0000 7f53c2655780 -1 Falling back to public interface 2026-02-19T16:27:08.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:27:08.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:08.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:27:08.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:08.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:08.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:08.265 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:08.265 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:08.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:08.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:27:08.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:08.633 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:08.630+0000 7f53c2655780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:27:09.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:09.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:09.493 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:09.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:09.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:09.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:27:09.720 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/919298613,v1:127.0.0.1:6819/919298613] [v2:127.0.0.1:6820/919298613,v1:127.0.0.1:6821/919298613] exists,up c552d47c-f028-42dd-aba4-9523cc4f84a1 2026-02-19T16:27:09.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:09.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:09.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:09.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:27:09.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:09.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:09.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2fda4442-8415-4c3d-b4f1-d54ab74351bf 2026-02-19T16:27:09.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2fda4442-8415-4c3d-b4f1-d54ab74351bf' 2026-02-19T16:27:09.728 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 2fda4442-8415-4c3d-b4f1-d54ab74351bf 2026-02-19T16:27:09.728 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:09.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDdOZdpDFalKxAAjYFhR0uKPCYwMVRsoxlBaA== 2026-02-19T16:27:09.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDdOZdpDFalKxAAjYFhR0uKPCYwMVRsoxlBaA=="}' 2026-02-19T16:27:09.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2fda4442-8415-4c3d-b4f1-d54ab74351bf -i td/test-erasure-eio/3/new.json 2026-02-19T16:27:09.965 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:27:09.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:27:09.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDdOZdpDFalKxAAjYFhR0uKPCYwMVRsoxlBaA== --osd-uuid 2fda4442-8415-4c3d-b4f1-d54ab74351bf 2026-02-19T16:27:09.990 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:09.989+0000 7faa89c38780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:09.992 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:09.990+0000 7faa89c38780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:09.993 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:09.991+0000 7faa89c38780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:09.993 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:09.991+0000 7faa89c38780 -1 bdev(0x3b3ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:09.994 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:09.992+0000 7faa89c38780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:27:10.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:27:10.327 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:10.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:27:10.328 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:27:10.328 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:10.645 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:27:10.646 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:27:10.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:10.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:10.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:10.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:10.660 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:10.658+0000 7fcbf5e41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:10.661 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:10.659+0000 7fcbf5e41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:10.662 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:10.660+0000 7fcbf5e41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:10.833 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:10.832+0000 7fcbf5e41780 -1 Falling back to public interface 2026-02-19T16:27:10.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:27:10.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:10.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:27:10.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:10.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:10.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:10.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:10.876 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:10.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:10.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:27:11.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:11.134 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:11.133+0000 7fcbf5e41780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:27:12.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:12.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:12.106 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:12.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:12.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:12.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:27:12.337 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1602775721,v1:127.0.0.1:6827/1602775721] [v2:127.0.0.1:6828/1602775721,v1:127.0.0.1:6829/1602775721] exists,up 2fda4442-8415-4c3d-b4f1-d54ab74351bf 2026-02-19T16:27:12.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:12.342 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:12.343 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:12.344 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c0a25b42-3617-4b9c-9ad9-111207f61659 2026-02-19T16:27:12.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 c0a25b42-3617-4b9c-9ad9-111207f61659' 2026-02-19T16:27:12.346 INFO:tasks.workunit.client.0.trial127.stdout:add osd4 c0a25b42-3617-4b9c-9ad9-111207f61659 2026-02-19T16:27:12.346 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:12.350 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgOZdpKY3MFBAAmqUavdJ+IOcErSfiWYyUBw== 2026-02-19T16:27:12.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgOZdpKY3MFBAAmqUavdJ+IOcErSfiWYyUBw=="}' 2026-02-19T16:27:12.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c0a25b42-3617-4b9c-9ad9-111207f61659 -i td/test-erasure-eio/4/new.json 2026-02-19T16:27:12.582 INFO:tasks.workunit.client.0.trial127.stdout:4 2026-02-19T16:27:12.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T16:27:12.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgOZdpKY3MFBAAmqUavdJ+IOcErSfiWYyUBw== --osd-uuid c0a25b42-3617-4b9c-9ad9-111207f61659 2026-02-19T16:27:12.608 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:12.606+0000 7f73c7864780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:12.609 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:12.608+0000 7f73c7864780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:12.610 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:12.609+0000 7f73c7864780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:12.610 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:12.609+0000 7f73c7864780 -1 bdev(0x26fec00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:12.610 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:12.609+0000 7f73c7864780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T16:27:12.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T16:27:12.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:12.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T16:27:12.929 INFO:tasks.workunit.client.0.trial127.stdout:adding osd4 key to auth repository 2026-02-19T16:27:12.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:13.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T16:27:13.244 INFO:tasks.workunit.client.0.trial127.stdout:start osd.4 2026-02-19T16:27:13.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:13.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:13.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:13.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:13.258 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:13.257+0000 7f712d844780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:13.260 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:13.258+0000 7f712d844780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:13.260 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:13.259+0000 7f712d844780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:13.440 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:13.438+0000 7f712d844780 -1 Falling back to public interface 2026-02-19T16:27:13.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T16:27:13.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:13.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T16:27:13.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:13.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:13.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:13.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:13.471 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:13.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:13.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:27:13.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:13.732 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:13.730+0000 7f712d844780 -1 osd.4 0 log_to_monitors true 2026-02-19T16:27:14.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:14.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:14.705 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:14.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:14.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:14.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T16:27:14.935 INFO:tasks.workunit.client.0.trial127.stdout:osd.4 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1799239305,v1:127.0.0.1:6835/1799239305] [v2:127.0.0.1:6836/1799239305,v1:127.0.0.1:6837/1799239305] exists,up c0a25b42-3617-4b9c-9ad9-111207f61659 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T16:27:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:14.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:14.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8c9fdfc4-da11-485c-ae39-4f5fc2ac5dee 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 8c9fdfc4-da11-485c-ae39-4f5fc2ac5dee' 2026-02-19T16:27:14.942 INFO:tasks.workunit.client.0.trial127.stdout:add osd5 8c9fdfc4-da11-485c-ae39-4f5fc2ac5dee 2026-02-19T16:27:14.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:14.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDiOZdpRcV3OBAAFhV8My4L4T/E/pxP1stjeA== 2026-02-19T16:27:14.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDiOZdpRcV3OBAAFhV8My4L4T/E/pxP1stjeA=="}' 2026-02-19T16:27:14.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8c9fdfc4-da11-485c-ae39-4f5fc2ac5dee -i td/test-erasure-eio/5/new.json 2026-02-19T16:27:15.181 INFO:tasks.workunit.client.0.trial127.stdout:5 2026-02-19T16:27:15.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T16:27:15.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDiOZdpRcV3OBAAFhV8My4L4T/E/pxP1stjeA== --osd-uuid 8c9fdfc4-da11-485c-ae39-4f5fc2ac5dee 2026-02-19T16:27:15.207 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.205+0000 7fbca342b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:15.208 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.206+0000 7fbca342b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:15.209 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.207+0000 7fbca342b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:15.209 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.207+0000 7fbca342b780 -1 bdev(0x3666c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:15.209 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.207+0000 7fbca342b780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T16:27:15.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T16:27:15.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:15.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T16:27:15.539 INFO:tasks.workunit.client.0.trial127.stdout:adding osd5 key to auth repository 2026-02-19T16:27:15.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:15.855 INFO:tasks.workunit.client.0.trial127.stdout:start osd.5 2026-02-19T16:27:15.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T16:27:15.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:15.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:15.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:15.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:15.871 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.869+0000 7fcb16039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:15.872 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.871+0000 7fcb16039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:15.874 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:15.872+0000 7fcb16039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:16.083 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:16.082+0000 7fcb16039780 -1 Falling back to public interface 2026-02-19T16:27:16.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T16:27:16.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:16.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T16:27:16.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:16.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:16.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:16.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:16.090 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:16.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:16.090 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:27:16.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:16.374 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:16.371+0000 7fcb16039780 -1 osd.5 0 log_to_monitors true 2026-02-19T16:27:17.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:17.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:17.316 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:17.316 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:17.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:17.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T16:27:17.542 INFO:tasks.workunit.client.0.trial127.stdout:osd.5 up in weight 1 up_from 43 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/393577475,v1:127.0.0.1:6843/393577475] [v2:127.0.0.1:6844/393577475,v1:127.0.0.1:6845/393577475] exists,up 8c9fdfc4-da11-485c-ae39-4f5fc2ac5dee 2026-02-19T16:27:17.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:17.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:17.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:17.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:17.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:17.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T16:27:17.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:17.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=47589593-6cd3-4182-9173-d560d4067b7a 2026-02-19T16:27:17.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 47589593-6cd3-4182-9173-d560d4067b7a' 2026-02-19T16:27:17.550 INFO:tasks.workunit.client.0.trial127.stdout:add osd6 47589593-6cd3-4182-9173-d560d4067b7a 2026-02-19T16:27:17.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:17.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDlOZdpx8oPIRAAFJO/CZ6/i2YM6QmCxpXLDQ== 2026-02-19T16:27:17.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDlOZdpx8oPIRAAFJO/CZ6/i2YM6QmCxpXLDQ=="}' 2026-02-19T16:27:17.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 47589593-6cd3-4182-9173-d560d4067b7a -i td/test-erasure-eio/6/new.json 2026-02-19T16:27:17.787 INFO:tasks.workunit.client.0.trial127.stdout:6 2026-02-19T16:27:17.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T16:27:17.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlOZdpx8oPIRAAFJO/CZ6/i2YM6QmCxpXLDQ== --osd-uuid 47589593-6cd3-4182-9173-d560d4067b7a 2026-02-19T16:27:17.811 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:17.809+0000 7fb652863780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:17.812 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:17.811+0000 7fb652863780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:17.813 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:17.811+0000 7fb652863780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:17.814 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:17.812+0000 7fb652863780 -1 bdev(0x2d48c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:17.814 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:17.812+0000 7fb652863780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T16:27:18.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T16:27:18.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:18.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T16:27:18.155 INFO:tasks.workunit.client.0.trial127.stdout:adding osd6 key to auth repository 2026-02-19T16:27:18.155 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:18.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T16:27:18.476 INFO:tasks.workunit.client.0.trial127.stdout:start osd.6 2026-02-19T16:27:18.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:18.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:18.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:18.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:18.491 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:18.489+0000 7ff952e2b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:18.492 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:18.490+0000 7ff952e2b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:18.493 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:18.491+0000 7ff952e2b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:18.650 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:18.648+0000 7ff952e2b780 -1 Falling back to public interface 2026-02-19T16:27:18.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T16:27:18.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:18.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T16:27:18.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:18.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:18.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:18.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:18.704 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:18.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:18.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:27:18.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:19.006 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:19.004+0000 7ff952e2b780 -1 osd.6 0 log_to_monitors true 2026-02-19T16:27:19.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:19.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:19.929 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:19.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:19.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:19.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T16:27:20.158 INFO:tasks.workunit.client.0.trial127.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2775512459,v1:127.0.0.1:6851/2775512459] [v2:127.0.0.1:6852/2775512459,v1:127.0.0.1:6853/2775512459] exists,up 47589593-6cd3-4182-9173-d560d4067b7a 2026-02-19T16:27:20.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:20.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:20.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:20.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:27:20.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:27:20.161 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:27:20.161 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:20.161 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:20.161 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:20.161 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:27:20.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:27:20.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:27:20.205 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:27:20.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:27:20.211 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:27:03.295+0000 7f3bc320e780 0 load: jerasure load: lrc 2026-02-19T16:27:20.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-19T16:27:20.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T16:27:20.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:27:20.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:27:20.213 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T16:27:20.213 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:27:20.213 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T16:27:20.213 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:27:20.213 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T16:27:20.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:27:20.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:27:20.865 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:27:20.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:27:21.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:27:21.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:27:21.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:27:21.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:27:21.876 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:27:21.877 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:27:21.877 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:27:21.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:27:21.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:27:21.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:27:21.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:27:21.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:27:21.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:27:21.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:27:21.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:27:21.962 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:27:22.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:27:22.191 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:22.191 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:27:22.191 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:27:22.191 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:27:22.191 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:27:22.191 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:27:22.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:27:22.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:22.192 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:27:22.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T16:27:22.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T16:27:22.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T16:27:22.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:22.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:27:22.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-19T16:27:22.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-19T16:27:22.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607559' 2026-02-19T16:27:22.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:22.326 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:27:22.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T16:27:22.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T16:27:22.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607559 2-77309411333' 2026-02-19T16:27:22.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:22.392 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:27:22.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-19T16:27:22.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-19T16:27:22.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607559 2-77309411333 3-111669149701' 2026-02-19T16:27:22.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:22.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:27:22.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822660 2026-02-19T16:27:22.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822660 2026-02-19T16:27:22.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607559 2-77309411333 3-111669149701 4-154618822660' 2026-02-19T16:27:22.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:22.520 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:27:22.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593733 2026-02-19T16:27:22.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593733 2026-02-19T16:27:22.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607559 2-77309411333 3-111669149701 4-154618822660 5-184683593733' 2026-02-19T16:27:22.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:22.584 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:27:22.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-19T16:27:22.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-19T16:27:22.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607559 2-77309411333 3-111669149701 4-154618822660 5-184683593733 6-214748364803' 2026-02-19T16:27:22.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:22.650 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-19T16:27:22.650 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:22.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:27:22.651 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T16:27:22.651 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:22.652 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-19T16:27:22.652 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T16:27:22.653 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803782 2026-02-19T16:27:22.653 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:22.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-19T16:27:22.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:27:23.877 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:27:23.878 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:24.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T16:27:24.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:24.108 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607559 2026-02-19T16:27:24.108 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:24.108 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:27:24.109 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-19T16:27:24.109 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:24.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607559 2026-02-19T16:27:24.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-19T16:27:24.111 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607559 2026-02-19T16:27:24.111 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:27:24.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-19T16:27:24.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:24.339 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-19T16:27:24.339 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:24.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:27:24.340 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T16:27:24.340 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:24.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-19T16:27:24.341 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T16:27:24.342 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411333 2026-02-19T16:27:24.342 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:27:24.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T16:27:24.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:24.571 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149701 2026-02-19T16:27:24.571 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:24.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:27:24.573 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-19T16:27:24.573 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:24.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149701 2026-02-19T16:27:24.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-19T16:27:24.574 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149701 2026-02-19T16:27:24.574 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:27:24.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-19T16:27:24.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:24.802 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822660 2026-02-19T16:27:24.803 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:24.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:27:24.804 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822660 2026-02-19T16:27:24.804 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:24.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822660 2026-02-19T16:27:24.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822660' 2026-02-19T16:27:24.805 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 154618822660 2026-02-19T16:27:24.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:27:25.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-19T16:27:25.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:25.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593733 2026-02-19T16:27:25.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:25.037 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:27:25.037 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593733 2026-02-19T16:27:25.037 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:25.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593733 2026-02-19T16:27:25.039 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593733' 2026-02-19T16:27:25.039 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 184683593733 2026-02-19T16:27:25.039 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:27:25.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593733 -lt 184683593733 2026-02-19T16:27:25.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:25.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364803 2026-02-19T16:27:25.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:25.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-19T16:27:25.280 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364803 2026-02-19T16:27:25.281 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:25.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364803 2026-02-19T16:27:25.282 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-19T16:27:25.282 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.6 seq 214748364803 2026-02-19T16:27:25.282 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T16:27:25.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364804 -lt 214748364803 2026-02-19T16:27:25.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:27:25.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:25.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:27:25.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:27:26.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:27:26.051 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:27:26.051 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:26.051 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:26.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:27:26.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:27:26.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:27:26.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:27:26.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:27:26.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:27:26.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T16:27:26.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-19T16:27:26.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:27:26.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:27:26.391 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-19T16:27:26.392 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:27:26.392 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:27:26.392 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:27:26.392 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:27:26.619 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:27:26.620 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T16:27:26.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:27:26.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T16:27:26.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:26.623 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:27:26.624 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:27:26.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:27:26.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:27:26.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:27:26.674 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-19T16:27:26.674 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:27:26.675 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:27:26.675 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:26.675 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:26.675 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:26.675 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:27:26.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:27:26.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-19T16:27:26.722 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-19T16:27:26.722 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:27:26.723 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T16:27:26.723 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:27:26.723 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-19T16:27:26.952 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-19T16:27:26.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-19T16:27:26.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-19T16:27:26.953 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:27:26.953 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:27:26.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:27:26.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:27:26.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:27:27.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:27:27.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-19T16:27:27.485 INFO:tasks.workunit.client.0.trial127.stderr:osd.6 is already down. 2026-02-19T16:27:27.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-19T16:27:27.769 INFO:tasks.workunit.client.0.trial127.stderr:osd.6 is already out. 2026-02-19T16:27:27.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-19T16:27:27.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:27:27.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:27:27.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:27:27.777 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:27:27.778 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:27:27.778 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:27:27.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:27:27.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:27:27.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:27:27.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:27:27.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:27:27.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:27:27.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:27:27.867 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:27:27.867 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:27:28.093 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:27:28.093 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:28.093 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:27:28.094 INFO:tasks.workunit.client.0.trial127.stderr:3 2026-02-19T16:27:28.094 INFO:tasks.workunit.client.0.trial127.stderr:4 2026-02-19T16:27:28.094 INFO:tasks.workunit.client.0.trial127.stderr:5 2026-02-19T16:27:28.094 INFO:tasks.workunit.client.0.trial127.stderr:6' 2026-02-19T16:27:28.094 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:27:28.094 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:28.094 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:27:28.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T16:27:28.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T16:27:28.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T16:27:28.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:28.157 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:27:28.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607562 2026-02-19T16:27:28.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607562 2026-02-19T16:27:28.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607562' 2026-02-19T16:27:28.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:28.225 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:27:28.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-19T16:27:28.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-19T16:27:28.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607562 2-77309411336' 2026-02-19T16:27:28.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:28.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:27:28.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149704 2026-02-19T16:27:28.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149704 2026-02-19T16:27:28.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607562 2-77309411336 3-111669149704' 2026-02-19T16:27:28.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:28.354 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-19T16:27:28.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822663 2026-02-19T16:27:28.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822663 2026-02-19T16:27:28.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607562 2-77309411336 3-111669149704 4-154618822663' 2026-02-19T16:27:28.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:28.418 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-19T16:27:28.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593736 2026-02-19T16:27:28.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593736 2026-02-19T16:27:28.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607562 2-77309411336 3-111669149704 4-154618822663 5-184683593736' 2026-02-19T16:27:28.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:28.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-19T16:27:28.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-19T16:27:28.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-19T16:27:28.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-19T16:27:28.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-19T16:27:28.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:28.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-19T16:27:28.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:28.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:27:28.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T16:27:28.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:28.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-19T16:27:28.532 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T16:27:28.532 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803785 2026-02-19T16:27:28.533 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:28.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-19T16:27:28.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:27:29.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:27:29.763 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:29.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T16:27:29.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:29.994 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607562 2026-02-19T16:27:29.995 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:29.995 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:27:29.996 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607562 2026-02-19T16:27:29.996 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:29.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607562 2026-02-19T16:27:29.997 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607562' 2026-02-19T16:27:29.997 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607562 2026-02-19T16:27:29.998 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:27:30.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607562 -lt 51539607562 2026-02-19T16:27:30.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:30.225 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-19T16:27:30.225 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:30.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:27:30.226 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-19T16:27:30.227 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:30.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-19T16:27:30.228 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-19T16:27:30.228 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411336 2026-02-19T16:27:30.229 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:27:30.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-19T16:27:30.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:30.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149704 2026-02-19T16:27:30.454 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:30.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:27:30.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149704 2026-02-19T16:27:30.456 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:30.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149704 2026-02-19T16:27:30.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149704' 2026-02-19T16:27:30.457 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149704 2026-02-19T16:27:30.457 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:27:30.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149704 -lt 111669149704 2026-02-19T16:27:30.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:30.689 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822663 2026-02-19T16:27:30.689 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:30.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-19T16:27:30.690 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822663 2026-02-19T16:27:30.690 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:30.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822663 2026-02-19T16:27:30.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822663' 2026-02-19T16:27:30.692 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.4 seq 154618822663 2026-02-19T16:27:30.692 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T16:27:30.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822663 -lt 154618822663 2026-02-19T16:27:30.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:30.919 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593736 2026-02-19T16:27:30.919 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:30.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-19T16:27:30.920 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593736 2026-02-19T16:27:30.921 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:30.922 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593736 2026-02-19T16:27:30.922 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593736' 2026-02-19T16:27:30.922 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.5 seq 184683593736 2026-02-19T16:27:30.922 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T16:27:31.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593736 -lt 184683593736 2026-02-19T16:27:31.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:27:31.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:31.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:31.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:27:31.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:27:31.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:27:31.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:27:31.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:27:31.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:27:31.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:27:31.469 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:27:31.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:27:31.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:27:31.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:31.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:27:32.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-19T16:27:32.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:27:32.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:27:32.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:27:32.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:27:32.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:27:32.318 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:27:32.325 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:27:32.601 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:27:32.610 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:27:32.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:27:32.721 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:27:32.722 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:27:32.722 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:27:32.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:27:32.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:27:32.724 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:27:32.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:32.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:27:32.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:27:32.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:32.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:27:32.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:27:32.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:27:32.750 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:27:32.750 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:32.750 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:32.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:27:32.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:27:32.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:27:32.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:27:32.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:27:32.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:27:32.751 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:27:32.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:27:32.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:27:32.754 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:27:32.754 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:27:32.755 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:27:32.755 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:27:32.756 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:27:32.756 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:27:32.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:32.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:27:32.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:27:32.757 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:27:32.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:27:32.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:27:32.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:27:32.760 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:27:32.760 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:32.760 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:32.760 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:27:32.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:27:32.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:27:32.761 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:27:32.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:27:32.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:32.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:32.763 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:27:32.764 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:27:32.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:27:32.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:27:32.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:27:32.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:27:32.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:27:32.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:27:32.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:27:32.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:27:32.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:27:32.765 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:27:32.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:27:32.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:32.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:32.785 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:32.785 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:32.785 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:32.785 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:32.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:27:32.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:27:32.811 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:32.812 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:32.813 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:32.813 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:27:32.813 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:27:32.813 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:27:32.856 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:27:32.856 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:27:32.856 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:27:32.856 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:27:32.856 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:27:32.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:27:32.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:27:32.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:27:32.858 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:32.858 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:32.858 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:32.858 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:27:32.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:27:32.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:27:32.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:27:32.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:27:32.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:27:32.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:27:32.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:27:32.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:27:32.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:27:33.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:27:33.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:33.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:33.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:33.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:33.001 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:33.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:33.002 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:27:33.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:27:33.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:27:33.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:27:33.112 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:27:33.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:27:34.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:27:34.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:27:34.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:27:34.120 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:34.120 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:34.120 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:34.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:27:34.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:27:34.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:27:34.161 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:27:34.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:27:34.167 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:27:32.799+0000 7f9651efbc00 0 load: jerasure load: lrc 2026-02-19T16:27:34.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-19T16:27:34.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-19T16:27:34.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-19T16:27:34.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T16:27:34.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:27:34.168 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T16:27:34.169 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:34.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:27:34.171 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:34.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1a5f3a86-130d-4164-b05c-e43ab14de1be 2026-02-19T16:27:34.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1a5f3a86-130d-4164-b05c-e43ab14de1be' 2026-02-19T16:27:34.172 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 1a5f3a86-130d-4164-b05c-e43ab14de1be 2026-02-19T16:27:34.172 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:34.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD2OZdpEczHChAA+BdCRfBlO1JE6FG6i32Fkg== 2026-02-19T16:27:34.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD2OZdpEczHChAA+BdCRfBlO1JE6FG6i32Fkg=="}' 2026-02-19T16:27:34.182 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1a5f3a86-130d-4164-b05c-e43ab14de1be -i td/test-erasure-eio/0/new.json 2026-02-19T16:27:34.284 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:34.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:27:34.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD2OZdpEczHChAA+BdCRfBlO1JE6FG6i32Fkg== --osd-uuid 1a5f3a86-130d-4164-b05c-e43ab14de1be 2026-02-19T16:27:34.307 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.305+0000 7f94c5832780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:34.308 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.305+0000 7f94c5832780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:34.309 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.307+0000 7f94c5832780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:34.309 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.308+0000 7f94c5832780 -1 bdev(0x40eac00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:34.309 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.308+0000 7f94c5832780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:27:34.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:27:34.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:34.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:27:34.711 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:27:34.711 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:34.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:27:34.814 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:27:34.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:34.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:34.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:34.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:34.830 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.828+0000 7fbb80e42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:34.831 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.830+0000 7fbb80e42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:34.832 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:34.831+0000 7fbb80e42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:34.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:34.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:27:35.008 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:35.047 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:35.046+0000 7fbb80e42780 -1 Falling back to public interface 2026-02-19T16:27:35.309 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:35.307+0000 7fbb80e42780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:27:36.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:36.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:36.009 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:36.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:36.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:36.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:27:36.243 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1731039895,v1:127.0.0.1:6803/1731039895] [v2:127.0.0.1:6804/1731039895,v1:127.0.0.1:6805/1731039895] exists,up 1a5f3a86-130d-4164-b05c-e43ab14de1be 2026-02-19T16:27:36.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:36.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:36.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:27:36.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:36.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:27:36.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2d6c1039-a418-4c84-b873-ae0882692e74' 2026-02-19T16:27:36.246 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:27:36.247 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:36.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD4OZdpsZM2DxAAjTPkYYLTur/tbOVOGXmmsA== 2026-02-19T16:27:36.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD4OZdpsZM2DxAAjTPkYYLTur/tbOVOGXmmsA=="}' 2026-02-19T16:27:36.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2d6c1039-a418-4c84-b873-ae0882692e74 -i td/test-erasure-eio/1/new.json 2026-02-19T16:27:36.489 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:36.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:27:36.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD4OZdpsZM2DxAAjTPkYYLTur/tbOVOGXmmsA== --osd-uuid 2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:27:36.514 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:36.512+0000 7fc25ca11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:36.516 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:36.514+0000 7fc25ca11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:36.516 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:36.515+0000 7fc25ca11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:36.517 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:36.515+0000 7fc25ca11780 -1 bdev(0x4152c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:36.517 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:36.515+0000 7fc25ca11780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:27:36.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:27:36.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:36.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:27:36.848 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:27:36.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:37.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:27:37.167 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:27:37.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:37.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:37.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:37.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:37.182 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:37.180+0000 7f3f37244780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:37.183 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:37.181+0000 7f3f37244780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:37.184 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:37.182+0000 7f3f37244780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:37.318 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:37.316+0000 7f3f37244780 -1 Falling back to public interface 2026-02-19T16:27:37.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:27:37.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:37.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:27:37.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:37.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:37.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:37.396 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:37.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:37.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:37.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:27:37.613 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:37.611+0000 7f3f37244780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:27:37.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:38.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:38.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:38.625 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:38.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:38.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:38.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:27:38.855 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2839066470,v1:127.0.0.1:6811/2839066470] [v2:127.0.0.1:6812/2839066470,v1:127.0.0.1:6813/2839066470] exists,up 2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:38.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:38.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:27:38.858 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:38.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5cd78c83-a636-421a-9a68-b84a3b6fc7c9 2026-02-19T16:27:38.859 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5cd78c83-a636-421a-9a68-b84a3b6fc7c9' 2026-02-19T16:27:38.859 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 5cd78c83-a636-421a-9a68-b84a3b6fc7c9 2026-02-19T16:27:38.859 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:38.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD6OZdpQLC5MxAAPAzyuQ1Lr24gVmftcZdjPg== 2026-02-19T16:27:38.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD6OZdpQLC5MxAAPAzyuQ1Lr24gVmftcZdjPg=="}' 2026-02-19T16:27:38.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5cd78c83-a636-421a-9a68-b84a3b6fc7c9 -i td/test-erasure-eio/2/new.json 2026-02-19T16:27:39.101 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:27:39.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:27:39.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD6OZdpQLC5MxAAPAzyuQ1Lr24gVmftcZdjPg== --osd-uuid 5cd78c83-a636-421a-9a68-b84a3b6fc7c9 2026-02-19T16:27:39.126 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.124+0000 7fa90ac24780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:39.127 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.126+0000 7fa90ac24780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:39.128 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.126+0000 7fa90ac24780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:39.128 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.127+0000 7fa90ac24780 -1 bdev(0x270ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:39.128 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.127+0000 7fa90ac24780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:27:39.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:27:39.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:39.459 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:27:39.459 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:27:39.460 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:39.793 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:27:39.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:27:39.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:39.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:39.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:39.794 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:39.809 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.807+0000 7ff925831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:39.810 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.808+0000 7ff925831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:39.811 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.809+0000 7ff925831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:39.942 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:39.940+0000 7ff925831780 -1 Falling back to public interface 2026-02-19T16:27:40.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:27:40.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:40.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:27:40.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:40.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:40.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:40.020 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:40.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:40.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:40.021 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:27:40.243 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:40.281 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:40.280+0000 7ff925831780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:27:41.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:41.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:41.245 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:41.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:41.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:41.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:27:41.476 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/37395619,v1:127.0.0.1:6819/37395619] [v2:127.0.0.1:6820/37395619,v1:127.0.0.1:6821/37395619] exists,up 5cd78c83-a636-421a-9a68-b84a3b6fc7c9 2026-02-19T16:27:41.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:41.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:41.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:41.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:41.482 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:41.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:27:41.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:27:41.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:27:41.484 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:27:41.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:27:41.484 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:27:41.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 db7a0eaf-4dbd-40d8-9891-10e55fede567' 2026-02-19T16:27:41.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:27:41.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD9OZdpowMqHRAAIKMia+roauWRK7dl9kHV6w== 2026-02-19T16:27:41.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD9OZdpowMqHRAAIKMia+roauWRK7dl9kHV6w=="}' 2026-02-19T16:27:41.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new db7a0eaf-4dbd-40d8-9891-10e55fede567 -i td/test-erasure-eio/3/new.json 2026-02-19T16:27:41.722 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:27:41.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:27:41.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD9OZdpowMqHRAAIKMia+roauWRK7dl9kHV6w== --osd-uuid db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:27:41.747 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:41.745+0000 7f8dea9d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:41.748 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:41.746+0000 7f8dea9d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:41.749 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:41.747+0000 7f8dea9d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:41.749 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:41.747+0000 7f8dea9d8780 -1 bdev(0x3406c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:27:41.749 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:41.747+0000 7f8dea9d8780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:27:42.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:27:42.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:27:42.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:27:42.076 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:27:42.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:27:42.395 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:27:42.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:27:42.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:42.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:27:42.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:27:42.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:27:42.410 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:42.408+0000 7f8ba8872780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:42.412 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:42.410+0000 7f8ba8872780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:42.412 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:42.411+0000 7f8ba8872780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:42.537 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:42.534+0000 7f8ba8872780 -1 Falling back to public interface 2026-02-19T16:27:42.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:27:42.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:42.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:27:42.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:42.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:42.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:42.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:42.620 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:27:42.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:42.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:27:42.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:42.905 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:42.903+0000 7f8ba8872780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:27:43.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:43.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:43.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:43.850 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:27:43.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:43.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:27:44.077 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/549520070,v1:127.0.0.1:6827/549520070] [v2:127.0.0.1:6828/549520070,v1:127.0.0.1:6829/549520070] exists,up db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:27:44.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:44.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:27:44.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:27:44.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:27:44.123 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:27:44.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:27:44.128 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:27:35.048+0000 7fbb80e42780 0 load: jerasure load: lrc 2026-02-19T16:27:44.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-19T16:27:44.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T16:27:44.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:27:44.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:27:44.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T16:27:44.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:27:44.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T16:27:44.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:27:44.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T16:27:44.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:27:44.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:27:44.792 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:27:44.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:27:45.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:27:45.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:27:45.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:27:45.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:27:45.802 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:27:45.803 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:27:45.804 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:27:45.804 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:27:45.804 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:27:45.804 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:27:45.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:27:45.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:27:45.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:27:45.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:27:45.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:27:45.891 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:27:46.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:27:46.125 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:46.125 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:27:46.125 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:27:46.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:27:46.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:46.125 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:27:46.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-19T16:27:46.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-19T16:27:46.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-19T16:27:46.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:46.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:27:46.252 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-19T16:27:46.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-19T16:27:46.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574853' 2026-02-19T16:27:46.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:46.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:27:46.318 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T16:27:46.318 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T16:27:46.318 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574853 2-77309411331' 2026-02-19T16:27:46.318 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:46.319 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:27:46.382 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-19T16:27:46.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-19T16:27:46.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574853 2-77309411331 3-115964116995' 2026-02-19T16:27:46.383 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:46.383 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-19T16:27:46.384 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:46.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:27:46.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:46.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-19T16:27:46.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-19T16:27:46.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-19T16:27:46.387 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803780 2026-02-19T16:27:46.387 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:46.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-19T16:27:46.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:27:47.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:27:47.615 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:47.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-19T16:27:47.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:47.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-19T16:27:47.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:47.846 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:27:47.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-19T16:27:47.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:47.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-19T16:27:47.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-19T16:27:47.848 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 55834574853 2026-02-19T16:27:47.849 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:27:48.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-19T16:27:48.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:48.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-19T16:27:48.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:48.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:27:48.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T16:27:48.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:48.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-19T16:27:48.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T16:27:48.081 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411331 2026-02-19T16:27:48.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:27:48.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T16:27:48.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:48.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116995 2026-02-19T16:27:48.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:48.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:27:48.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-19T16:27:48.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:48.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116995 2026-02-19T16:27:48.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-19T16:27:48.314 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 115964116995 2026-02-19T16:27:48.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:27:48.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-19T16:27:48.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:27:48.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:48.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:48.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:27:48.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:27:48.864 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:27:48.864 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:27:48.864 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:27:48.864 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:27:48.865 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:27:48.865 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:27:49.092 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:27:49.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:27:49.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:49.093 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-95545-0-10 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-95545-0-10 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-95545-0-10 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:27:49.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-95545-0-10 td/test-erasure-eio/ORIGINAL 2026-02-19T16:27:49.430 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-95545-0-10 td/test-erasure-eio 0 10 set 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-0-10 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-0-10 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-0-10 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-0-10 2026-02-19T16:27:49.431 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:27:49.663 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:27:49.663 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:49.663 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:27:49.664 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:27:49.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:27:49.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:27:49.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-19T16:27:49.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:27:49.944 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:27:49.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:27:49.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T16:27:49.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T16:27:49.953 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:27:49.953 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:27:49.953 INFO:tasks.workunit.client.0.trial127.stderr:10 bytes copied, 3.7069e-05 s, 270 kB/s 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T16:27:49.954 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:27:49.955 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:27:49.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:27:49.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:27:49.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T16:27:50.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:50.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T16:27:50.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:27:50.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:50.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:start osd.3 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T16:27:50.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T16:27:50.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:27:50.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:27:50.405 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:27:50.415 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:50.413+0000 7f0fd1a12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:50.416 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:50.414+0000 7f0fd1a12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:50.417 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:50.415+0000 7f0fd1a12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:50.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:50.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:27:50.638 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:50.637+0000 7f0fd1a12780 -1 Falling back to public interface 2026-02-19T16:27:50.853 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:51.005 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:51.003+0000 7f0fd1a12780 -1 osd.3 35 log_to_monitors true 2026-02-19T16:27:51.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:51.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:51.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:51.855 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:51.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:51.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:27:52.083 INFO:tasks.workunit.client.0.trial127.stderr:osd.3 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [27,35) [v2:127.0.0.1:6826/2563100927,v1:127.0.0.1:6827/2563100927] [v2:127.0.0.1:6828/2563100927,v1:127.0.0.1:6829/2563100927] exists,up db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:27:52.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:52.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:52.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:52.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:27:52.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:27:52.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:27:52.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:27:52.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:27:52.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:27:52.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:27:52.170 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:27:52.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:27:52.399 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:52.399 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:27:52.399 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:27:52.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:27:52.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:52.399 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:27:52.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T16:27:52.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T16:27:52.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T16:27:52.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:52.464 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:27:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-19T16:27:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-19T16:27:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574856' 2026-02-19T16:27:52.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:52.529 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:27:52.593 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-19T16:27:52.593 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-19T16:27:52.593 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-77309411335' 2026-02-19T16:27:52.593 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:52.593 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:27:52.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691843 2026-02-19T16:27:52.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691843 2026-02-19T16:27:52.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-77309411335 3-171798691843' 2026-02-19T16:27:52.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:52.658 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-19T16:27:52.658 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:52.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:27:52.659 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T16:27:52.659 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:52.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-19T16:27:52.661 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T16:27:52.661 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803784 2026-02-19T16:27:52.661 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:52.893 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-19T16:27:52.893 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:27:53.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:27:53.895 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:54.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T16:27:54.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:54.122 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574856 2026-02-19T16:27:54.122 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:54.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:27:54.123 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574856 2026-02-19T16:27:54.124 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:54.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-19T16:27:54.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574856' 2026-02-19T16:27:54.125 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 55834574856 2026-02-19T16:27:54.125 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:27:54.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-19T16:27:54.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:54.354 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411335 2026-02-19T16:27:54.354 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:54.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:27:54.355 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-19T16:27:54.356 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:54.357 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411335 2026-02-19T16:27:54.357 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-19T16:27:54.357 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411335 2026-02-19T16:27:54.357 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:27:54.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-19T16:27:54.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:54.582 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-171798691843 2026-02-19T16:27:54.582 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:54.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:27:54.583 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-171798691843 2026-02-19T16:27:54.583 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:54.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=171798691843 2026-02-19T16:27:54.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 171798691843' 2026-02-19T16:27:54.584 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 171798691843 2026-02-19T16:27:54.585 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:27:54.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 171798691843 -lt 171798691843 2026-02-19T16:27:54.811 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:27:54.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:54.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:55.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:27:55.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:27:55.129 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:27:55.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:27:55.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:27:55.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:27:55.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:27:55.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:27:55.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:27:55.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:27:55.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:27:55.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:27:55.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:27:55.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:27:55.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:27:55.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:27:55.679 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:27:55.992 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:27:56.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-0-10 2026-02-19T16:27:56.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:27:56.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:27:56.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-0-10 2026-02-19T16:27:56.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:27:56.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:27:56.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-0-10 td/test-erasure-eio/COPY 2026-02-19T16:27:56.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:27:56.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:27:56.021 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-19T16:27:56.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-19T16:27:56.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-95545-0-10 td/test-erasure-eio 1 10 set 2026-02-19T16:27:56.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-0-10 2026-02-19T16:27:56.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-0-10 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:27:56.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-0-10 2026-02-19T16:27:56.026 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-0-10 2026-02-19T16:27:56.026 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:27:56.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:27:56.253 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:56.253 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:27:56.253 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:27:56.253 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:27:56.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:27:56.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T16:27:56.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:27:56.567 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:27:56.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:27:56.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T16:27:56.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:10 bytes copied, 3.5566e-05 s, 281 kB/s 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:27:56.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:27:56.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:27:56.887 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:27:56.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:56.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:27:56.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:27:56.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:27:56.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:27:56.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:27:56.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:27:57.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:27:57.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:27:57.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:27:57.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:27:57.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:27:57.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:27:57.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:27:57.047 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:27:57.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:27:57.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:27:57.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:27:57.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:27:57.049 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:27:57.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:27:57.049 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:27:57.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:27:57.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:27:57.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:27:57.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:27:57.059 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:57.057+0000 7ff59380d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:57.061 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:57.058+0000 7ff59380d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:57.062 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:57.059+0000 7ff59380d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:27:57.210 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:57.207+0000 7ff59380d780 -1 Falling back to public interface 2026-02-19T16:27:57.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:27:57.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:27:57.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:27:57.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:27:57.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:27:57.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:57.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:27:57.276 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:27:57.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:57.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:27:57.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:27:57.532 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:27:57.530+0000 7ff59380d780 -1 osd.1 46 log_to_monitors true 2026-02-19T16:27:58.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:27:58.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:27:58.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:27:58.503 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:58.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:27:58.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:27:58.729 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 47 last_clean_interval [13,46) [v2:127.0.0.1:6810/2709279268,v1:127.0.0.1:6811/2709279268] [v2:127.0.0.1:6812/2709279268,v1:127.0.0.1:6813/2709279268] exists,up 2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:27:58.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:27:58.730 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:27:58.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:27:58.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:27:58.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:27:58.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:27:58.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:27:58.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:27:58.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:27:58.816 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:27:58.816 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:27:59.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:27:59.043 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:27:59.043 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:27:59.044 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:27:59.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:27:59.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:59.044 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:27:59.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-19T16:27:59.109 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-19T16:27:59.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-19T16:27:59.110 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:59.110 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:27:59.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-19T16:27:59.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-19T16:27:59.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-214748364803' 2026-02-19T16:27:59.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:59.175 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:27:59.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411338 2026-02-19T16:27:59.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411338 2026-02-19T16:27:59.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-214748364803 2-77309411338' 2026-02-19T16:27:59.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:27:59.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:27:59.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691846 2026-02-19T16:27:59.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691846 2026-02-19T16:27:59.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-214748364803 2-77309411338 3-171798691846' 2026-02-19T16:27:59.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:27:59.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-19T16:27:59.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:27:59.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:27:59.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-19T16:27:59.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:27:59.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-19T16:27:59.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-19T16:27:59.307 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803787 2026-02-19T16:27:59.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:27:59.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-19T16:27:59.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:00.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:28:00.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:00.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-19T16:28:00.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:01.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:28:01.768 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:01.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-19T16:28:01.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:01.999 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-214748364803 2026-02-19T16:28:01.999 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:01.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:02.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-214748364803 2026-02-19T16:28:02.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:02.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364803 2026-02-19T16:28:02.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 214748364803' 2026-02-19T16:28:02.001 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 214748364803 2026-02-19T16:28:02.002 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:02.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-19T16:28:02.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:02.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411338 2026-02-19T16:28:02.240 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:02.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:02.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411338 2026-02-19T16:28:02.241 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:02.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411338 2026-02-19T16:28:02.242 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411338' 2026-02-19T16:28:02.242 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411338 2026-02-19T16:28:02.243 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:28:02.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-19T16:28:02.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:02.471 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-171798691846 2026-02-19T16:28:02.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:02.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:28:02.473 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-171798691846 2026-02-19T16:28:02.473 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:02.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=171798691846 2026-02-19T16:28:02.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 171798691846' 2026-02-19T16:28:02.474 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 171798691846 2026-02-19T16:28:02.474 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:02.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 171798691847 -lt 171798691846 2026-02-19T16:28:02.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:28:02.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:02.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:03.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:28:03.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:28:03.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:28:03.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:28:03.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:28:03.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:28:03.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:28:03.034 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:28:03.255 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:28:03.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:28:03.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:03.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:28:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:28:03.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:28:03.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:28:03.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:28:03.890 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:28:03.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T16:28:03.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-0-10 fail 2026-02-19T16:28:03.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:28:03.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:28:03.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-0-10 2026-02-19T16:28:03.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T16:28:03.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T16:28:03.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-95545-0-10 td/test-erasure-eio/COPY 2026-02-19T16:28:03.916 INFO:tasks.workunit.client.0.trial127.stderr:error getting pool-jerasure/obj-size-95545-0-10: (5) Input/output error 2026-02-19T16:28:03.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T16:28:03.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-95545-0-0 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-95545-0-0 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-95545-0-0 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:03.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:28:03.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:03.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:28:03.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-95545-0-0 td/test-erasure-eio/ORIGINAL 2026-02-19T16:28:03.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-95545-0-0 td/test-erasure-eio 0 0 set 2026-02-19T16:28:03.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-0-0 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:28:03.939 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-0-0 2026-02-19T16:28:03.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:28:03.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-0-0 2026-02-19T16:28:03.940 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-0-0 2026-02-19T16:28:03.940 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-19T16:28:04.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:28:04.484 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:28:04.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:28:04.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T16:28:04.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T16:28:04.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:04.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:28:04.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:28:04.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T16:28:04.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:04.497 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:04.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:04.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:04.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:28:04.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:28:04.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T16:28:04.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:28:04.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T16:28:04.802 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:04.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:04.947 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T16:28:04.948 INFO:tasks.workunit.client.0.trial127.stderr:start osd.3 2026-02-19T16:28:04.949 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:04.949 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T16:28:04.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T16:28:04.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:28:04.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:28:04.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:28:04.968 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:04.965+0000 7f2c90e38780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:04.969 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:04.967+0000 7f2c90e38780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:04.970 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:04.967+0000 7f2c90e38780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:05.106 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:05.103+0000 7f2c90e38780 -1 Falling back to public interface 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:05.183 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:28:05.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:05.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:05.410 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:05.444 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:05.441+0000 7f2c90e38780 -1 osd.3 54 log_to_monitors true 2026-02-19T16:28:06.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:06.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:06.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:06.413 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:06.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:06.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:06.640 INFO:tasks.workunit.client.0.trial127.stderr:osd.3 up in weight 1 up_from 60 up_thru 60 down_at 57 last_clean_interval [40,54) [v2:127.0.0.1:6826/3391054854,v1:127.0.0.1:6827/3391054854] [v2:127.0.0.1:6828/3391054854,v1:127.0.0.1:6829/3391054854] exists,up db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:28:06.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:06.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:06.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:06.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:28:06.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:28:06.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:28:06.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:28:06.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:28:06.641 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:28:06.641 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:28:06.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:28:06.642 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:28:06.642 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:28:06.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:28:06.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:28:06.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:28:06.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:28:06.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:28:06.729 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:28:06.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:28:06.956 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:06.956 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:28:06.956 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:28:06.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:28:06.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:06.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:28:07.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-19T16:28:07.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-19T16:28:07.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-19T16:28:07.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:07.020 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:28:07.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364806 2026-02-19T16:28:07.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364806 2026-02-19T16:28:07.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-214748364806' 2026-02-19T16:28:07.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:07.088 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:28:07.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411342 2026-02-19T16:28:07.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411342 2026-02-19T16:28:07.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-214748364806 2-77309411342' 2026-02-19T16:28:07.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:07.152 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:28:07.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037763 2026-02-19T16:28:07.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037763 2026-02-19T16:28:07.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-214748364806 2-77309411342 3-257698037763' 2026-02-19T16:28:07.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:07.217 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-19T16:28:07.217 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:07.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:28:07.218 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-19T16:28:07.218 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:07.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-19T16:28:07.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-19T16:28:07.219 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803791 2026-02-19T16:28:07.220 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:07.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-19T16:28:07.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:07.446 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-214748364806 2026-02-19T16:28:07.447 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:07.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:07.448 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-214748364806 2026-02-19T16:28:07.448 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:07.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364806 2026-02-19T16:28:07.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 214748364806' 2026-02-19T16:28:07.449 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 214748364806 2026-02-19T16:28:07.450 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:07.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364804 -lt 214748364806 2026-02-19T16:28:07.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:08.678 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:28:08.678 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:08.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364804 -lt 214748364806 2026-02-19T16:28:08.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:09.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:28:09.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:10.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364807 -lt 214748364806 2026-02-19T16:28:10.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:10.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411342 2026-02-19T16:28:10.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:10.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:10.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411342 2026-02-19T16:28:10.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:10.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411342 2026-02-19T16:28:10.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411342' 2026-02-19T16:28:10.142 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411342 2026-02-19T16:28:10.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:28:10.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411342 -lt 77309411342 2026-02-19T16:28:10.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:10.373 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-257698037763 2026-02-19T16:28:10.373 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:10.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:28:10.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-257698037763 2026-02-19T16:28:10.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:10.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=257698037763 2026-02-19T16:28:10.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 257698037763' 2026-02-19T16:28:10.376 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 257698037763 2026-02-19T16:28:10.376 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:10.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037763 -lt 257698037763 2026-02-19T16:28:10.607 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:28:10.608 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:10.609 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:28:10.923 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:28:11.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:28:11.145 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:28:11.145 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:11.146 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:11.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:28:11.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:28:11.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:28:11.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:28:11.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:28:11.777 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:28:11.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-0-0 2026-02-19T16:28:11.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:28:11.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:28:11.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-0-0 2026-02-19T16:28:11.786 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:28:11.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:28:11.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-0-0 td/test-erasure-eio/COPY 2026-02-19T16:28:11.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:28:11.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:28:11.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-95545-0-0 td/test-erasure-eio 1 0 set 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-0-0 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T16:28:11.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:28:11.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:28:11.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:28:11.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-0-0 2026-02-19T16:28:11.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:28:11.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-0-0 2026-02-19T16:28:11.810 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-0-0 2026-02-19T16:28:11.810 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T16:28:12.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:28:12.360 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:28:12.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:28:12.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T16:28:12.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:12.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:12.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:28:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:28:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:28:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:28:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:28:12.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:12.807 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:12.808 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:28:12.809 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:28:12.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:12.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:28:12.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:28:12.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:28:12.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:28:12.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:28:12.824 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:12.823+0000 7fa197050780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:12.826 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:12.824+0000 7fa197050780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:12.827 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:12.825+0000 7fa197050780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:12.951 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:12.949+0000 7fa197050780 -1 Falling back to public interface 2026-02-19T16:28:13.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:28:13.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:13.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:13.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:13.321 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:13.320+0000 7fa197050780 -1 osd.1 65 log_to_monitors true 2026-02-19T16:28:14.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:14.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:14.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:14.269 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:14.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:14.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:14.498 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 69 up_thru 69 down_at 66 last_clean_interval [50,65) [v2:127.0.0.1:6810/3158111253,v1:127.0.0.1:6811/3158111253] [v2:127.0.0.1:6812/3158111253,v1:127.0.0.1:6813/3158111253] exists,up 2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:28:14.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:14.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:14.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:14.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:28:14.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:28:14.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:28:14.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:28:14.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:28:14.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:28:14.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:28:14.586 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:28:14.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:28:14.809 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:14.810 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:28:14.810 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:28:14.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:28:14.810 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:14.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:28:14.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-19T16:28:14.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-19T16:28:14.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794' 2026-02-19T16:28:14.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:14.874 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:28:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743427 2026-02-19T16:28:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743427 2026-02-19T16:28:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-296352743427' 2026-02-19T16:28:14.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:14.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:28:15.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411345 2026-02-19T16:28:15.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411345 2026-02-19T16:28:15.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-296352743427 2-77309411345' 2026-02-19T16:28:15.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:15.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:28:15.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037766 2026-02-19T16:28:15.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037766 2026-02-19T16:28:15.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-296352743427 2-77309411345 3-257698037766' 2026-02-19T16:28:15.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:15.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803794 2026-02-19T16:28:15.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:15.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:28:15.071 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-19T16:28:15.071 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:15.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803794 2026-02-19T16:28:15.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-19T16:28:15.072 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803794 2026-02-19T16:28:15.072 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:15.303 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-19T16:28:15.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:15.304 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-296352743427 2026-02-19T16:28:15.304 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:15.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:15.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-296352743427 2026-02-19T16:28:15.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:15.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=296352743427 2026-02-19T16:28:15.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 296352743427' 2026-02-19T16:28:15.307 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 296352743427 2026-02-19T16:28:15.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:15.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 296352743427 -lt 296352743427 2026-02-19T16:28:15.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:15.536 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411345 2026-02-19T16:28:15.536 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:15.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:15.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411345 2026-02-19T16:28:15.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:15.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411345 2026-02-19T16:28:15.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411345' 2026-02-19T16:28:15.538 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411345 2026-02-19T16:28:15.539 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:28:15.772 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411345 -lt 77309411345 2026-02-19T16:28:15.772 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:15.772 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-257698037766 2026-02-19T16:28:15.772 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:15.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:28:15.774 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-257698037766 2026-02-19T16:28:15.774 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:15.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=257698037766 2026-02-19T16:28:15.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 257698037766' 2026-02-19T16:28:15.775 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 257698037766 2026-02-19T16:28:15.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:16.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037764 -lt 257698037766 2026-02-19T16:28:16.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:17.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:28:17.006 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:17.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037767 -lt 257698037766 2026-02-19T16:28:17.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:28:17.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:17.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:17.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:28:17.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:28:17.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:28:17.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:28:17.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:28:17.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:28:17.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:28:17.557 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:28:17.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:28:17.782 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:28:17.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:17.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:18.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:28:18.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:28:18.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:28:18.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:28:18.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:28:18.411 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:28:18.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T16:28:18.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-0-0 fail 2026-02-19T16:28:18.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:28:18.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:28:18.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-0-0 2026-02-19T16:28:18.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T16:28:18.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T16:28:18.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-95545-0-0 td/test-erasure-eio/COPY 2026-02-19T16:28:18.440 INFO:tasks.workunit.client.0.trial127.stderr:error getting pool-jerasure/obj-size-95545-0-0: (5) Input/output error 2026-02-19T16:28:18.442 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T16:28:18.442 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:28:18.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-19T16:28:18.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T16:28:18.444 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-95545-0-256 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-95545-0-256 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-95545-0-256 2026-02-19T16:28:18.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:28:18.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-95545-0-256 td/test-erasure-eio/ORIGINAL 2026-02-19T16:28:18.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-95545-0-256 td/test-erasure-eio 0 256 add 2026-02-19T16:28:18.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-0-256 2026-02-19T16:28:18.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:28:18.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:28:18.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-0-256 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-0-256 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-0-256 2026-02-19T16:28:18.463 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:28:18.692 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:28:18.693 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:18.693 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:28:18.693 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:28:18.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:28:18.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:28:18.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-19T16:28:18.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:28:18.974 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:28:18.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:28:18.984 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T16:28:18.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:18.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:18.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:18.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:18.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:19.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:19.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:19.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:28:19.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:28:19.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T16:28:19.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:28:19.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T16:28:19.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:28:19.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:19.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:start osd.3 2026-02-19T16:28:19.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:19.379 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T16:28:19.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T16:28:19.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:28:19.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:28:19.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:28:19.393 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:19.391+0000 7fbd7bc4d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:19.394 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:19.393+0000 7fbd7bc4d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:19.395 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:19.394+0000 7fbd7bc4d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:19.533 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:19.532+0000 7fbd7bc4d780 -1 Falling back to public interface 2026-02-19T16:28:19.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:19.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:19.834 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:19.866 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:19.864+0000 7fbd7bc4d780 -1 osd.3 72 log_to_monitors true 2026-02-19T16:28:20.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:20.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:20.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:20.835 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:20.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:20.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:21.067 INFO:tasks.workunit.client.0.trial127.stderr:osd.3 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [60,72) [v2:127.0.0.1:6826/1847320109,v1:127.0.0.1:6827/1847320109] [v2:127.0.0.1:6828/1847320109,v1:127.0.0.1:6829/1847320109] exists,up db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:28:21.067 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:28:21.068 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:28:21.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:28:21.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:28:21.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:28:21.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:28:21.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:28:21.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:28:21.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:28:21.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:28:21.157 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:28:21.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:28:21.384 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:21.384 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:28:21.384 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:28:21.384 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:28:21.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:21.385 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:28:21.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803797 2026-02-19T16:28:21.451 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803797 2026-02-19T16:28:21.451 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797' 2026-02-19T16:28:21.451 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:21.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:28:21.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743430 2026-02-19T16:28:21.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743430 2026-02-19T16:28:21.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-296352743430' 2026-02-19T16:28:21.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:21.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:28:21.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411349 2026-02-19T16:28:21.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411349 2026-02-19T16:28:21.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-296352743430 2-77309411349' 2026-02-19T16:28:21.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:21.579 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:28:21.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449091 2026-02-19T16:28:21.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449091 2026-02-19T16:28:21.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-296352743430 2-77309411349 3-335007449091' 2026-02-19T16:28:21.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:21.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803797 2026-02-19T16:28:21.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:21.649 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:28:21.649 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803797 2026-02-19T16:28:21.650 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:21.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803797 2026-02-19T16:28:21.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-19T16:28:21.651 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803797 2026-02-19T16:28:21.651 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:21.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-19T16:28:21.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:22.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:28:22.874 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:23.102 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-19T16:28:23.102 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:24.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:28:24.103 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:24.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803798 -lt 25769803797 2026-02-19T16:28:24.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:24.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-296352743430 2026-02-19T16:28:24.331 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:24.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:24.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-296352743430 2026-02-19T16:28:24.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:24.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=296352743430 2026-02-19T16:28:24.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 296352743430' 2026-02-19T16:28:24.334 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 296352743430 2026-02-19T16:28:24.334 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:24.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 296352743430 -lt 296352743430 2026-02-19T16:28:24.561 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:24.562 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411349 2026-02-19T16:28:24.562 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:24.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:24.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411349 2026-02-19T16:28:24.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:24.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411349 2026-02-19T16:28:24.564 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411349' 2026-02-19T16:28:24.564 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411349 2026-02-19T16:28:24.565 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:28:24.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411349 -lt 77309411349 2026-02-19T16:28:24.792 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:24.792 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-335007449091 2026-02-19T16:28:24.793 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:24.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:28:24.794 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-335007449091 2026-02-19T16:28:24.794 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:24.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449091 2026-02-19T16:28:24.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 335007449091' 2026-02-19T16:28:24.795 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 335007449091 2026-02-19T16:28:24.795 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:25.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-19T16:28:25.023 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:28:25.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:25.024 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:25.337 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:28:25.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:28:25.338 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:28:25.338 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:28:25.338 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:28:25.338 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:28:25.338 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:28:25.339 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:28:25.560 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:28:25.560 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:28:25.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:25.561 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:25.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:28:25.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:28:25.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:28:25.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T16:28:25.882 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:28:25.882 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:28:25.882 INFO:tasks.workunit.client.0.trial127.stderr:256 bytes copied, 1.9927e-05 s, 12.8 MB/s 2026-02-19T16:28:25.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:25.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:28:25.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:25.883 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:25.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:25.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:25.884 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T16:28:25.983 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:26.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:26.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:26.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:26.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:26.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:26.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:28:26.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:28:26.120 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr:start osd.3 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:28:26.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:28:26.133 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:26.131+0000 7f3efe9e1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:26.135 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:26.133+0000 7f3efe9e1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:26.135 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:26.134+0000 7f3efe9e1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:26.304 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:26.301+0000 7f3efe9e1780 -1 Falling back to public interface 2026-02-19T16:28:26.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:26.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:26.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:26.628 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:26.626+0000 7f3efe9e1780 -1 osd.3 80 log_to_monitors true 2026-02-19T16:28:27.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:27.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:27.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:27.580 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:27.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:27.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:27.805 INFO:tasks.workunit.client.0.trial127.stderr:osd.3 up in weight 1 up_from 84 up_thru 84 down_at 81 last_clean_interval [78,80) [v2:127.0.0.1:6826/493512986,v1:127.0.0.1:6827/493512986] [v2:127.0.0.1:6828/493512986,v1:127.0.0.1:6829/493512986] exists,up db7a0eaf-4dbd-40d8-9891-10e55fede567 2026-02-19T16:28:27.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:27.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:27.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:28:27.806 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:28:27.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:28:27.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:28:27.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:28:27.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:28:27.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:28:27.895 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:28:28.117 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:28:28.118 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:28.118 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:28:28.118 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:28:28.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:28:28.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:28.118 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:28:28.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803801 2026-02-19T16:28:28.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803801 2026-02-19T16:28:28.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801' 2026-02-19T16:28:28.186 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:28.186 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:28:28.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743433 2026-02-19T16:28:28.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743433 2026-02-19T16:28:28.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-296352743433' 2026-02-19T16:28:28.248 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:28.248 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:28:28.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411352 2026-02-19T16:28:28.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411352 2026-02-19T16:28:28.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-296352743433 2-77309411352' 2026-02-19T16:28:28.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:28.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:28:28.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=360777252867 2026-02-19T16:28:28.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 360777252867 2026-02-19T16:28:28.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-296352743433 2-77309411352 3-360777252867' 2026-02-19T16:28:28.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:28.376 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803801 2026-02-19T16:28:28.376 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:28.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:28:28.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803801 2026-02-19T16:28:28.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:28.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803801 2026-02-19T16:28:28.378 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803801' 2026-02-19T16:28:28.378 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803801 2026-02-19T16:28:28.379 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:28.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803799 -lt 25769803801 2026-02-19T16:28:28.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:29.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:28:29.606 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:29.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803801 -lt 25769803801 2026-02-19T16:28:29.834 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:29.834 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-296352743433 2026-02-19T16:28:29.834 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:29.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:29.836 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-296352743433 2026-02-19T16:28:29.836 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:29.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=296352743433 2026-02-19T16:28:29.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 296352743433' 2026-02-19T16:28:29.837 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 296352743433 2026-02-19T16:28:29.837 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:30.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 296352743434 -lt 296352743433 2026-02-19T16:28:30.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:30.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411352 2026-02-19T16:28:30.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:30.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:30.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411352 2026-02-19T16:28:30.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:30.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411352 2026-02-19T16:28:30.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411352' 2026-02-19T16:28:30.064 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411352 2026-02-19T16:28:30.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:28:30.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411352 -lt 77309411352 2026-02-19T16:28:30.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:30.289 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-360777252867 2026-02-19T16:28:30.289 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:30.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:28:30.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-360777252867 2026-02-19T16:28:30.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:30.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=360777252867 2026-02-19T16:28:30.292 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 360777252867' 2026-02-19T16:28:30.292 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 360777252867 2026-02-19T16:28:30.292 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:30.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 360777252867 -lt 360777252867 2026-02-19T16:28:30.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:28:30.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:30.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:30.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:28:30.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:28:30.837 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:28:30.837 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:28:30.837 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:28:30.837 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:28:30.837 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:28:30.838 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:28:31.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:28:31.061 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:28:31.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:31.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:31.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:28:31.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:28:31.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:28:31.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:28:31.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:28:31.690 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:28:31.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-0-256 2026-02-19T16:28:31.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:28:31.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:28:31.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-0-256 2026-02-19T16:28:31.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:28:31.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:28:31.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-0-256 td/test-erasure-eio/COPY 2026-02-19T16:28:31.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:28:31.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:28:31.720 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-95545-0-256 td/test-erasure-eio 1 256 add 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-0-256 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:28:31.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-0-256 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-0-256 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-0-256 2026-02-19T16:28:31.725 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:28:31.950 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:28:31.950 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:31.950 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:28:31.950 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:28:31.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:28:31.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:28:31.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T16:28:31.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:28:32.295 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:28:32.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T16:28:32.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:28:32.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:32.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:32.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:32.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:32.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:32.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:32.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:32.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:28:32.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:28:32.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:28:32.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:28:32.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:28:32.612 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:32.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:28:32.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:28:32.718 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:32.719 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:32.720 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:28:32.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:28:32.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:28:32.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:28:32.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:28:32.731 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:32.728+0000 7f684e22d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:32.732 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:32.729+0000 7f684e22d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:32.733 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:32.730+0000 7f684e22d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:32.879 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:32.876+0000 7f684e22d780 -1 Falling back to public interface 2026-02-19T16:28:32.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:28:32.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:32.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:33.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:33.184 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:33.181+0000 7f684e22d780 -1 osd.1 88 log_to_monitors true 2026-02-19T16:28:34.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:34.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:34.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:34.174 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:34.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:34.174 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:34.403 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 94 up_thru 94 down_at 91 last_clean_interval [69,88) [v2:127.0.0.1:6810/727273542,v1:127.0.0.1:6811/727273542] [v2:127.0.0.1:6812/727273542,v1:127.0.0.1:6813/727273542] exists,up 2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:28:34.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:28:34.404 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:28:34.490 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:28:34.490 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:28:34.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:28:34.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:28:34.491 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:28:34.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:28:34.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:28:34.716 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:34.716 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:28:34.716 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:28:34.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:28:34.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:34.716 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:28:34.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-19T16:28:34.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-19T16:28:34.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804' 2026-02-19T16:28:34.787 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:34.787 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:28:34.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=403726925827 2026-02-19T16:28:34.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 403726925827 2026-02-19T16:28:34.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-403726925827' 2026-02-19T16:28:34.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:34.856 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:28:34.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411355 2026-02-19T16:28:34.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411355 2026-02-19T16:28:34.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-403726925827 2-77309411355' 2026-02-19T16:28:34.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:34.921 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:28:34.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=360777252870 2026-02-19T16:28:34.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 360777252870 2026-02-19T16:28:34.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-403726925827 2-77309411355 3-360777252870' 2026-02-19T16:28:34.985 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:34.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803804 2026-02-19T16:28:34.985 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:34.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:28:34.987 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-19T16:28:34.987 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:34.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803804 2026-02-19T16:28:34.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-19T16:28:34.988 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803804 2026-02-19T16:28:34.988 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-19T16:28:35.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:35.219 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-403726925827 2026-02-19T16:28:35.219 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:35.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:35.220 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-403726925827 2026-02-19T16:28:35.221 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:35.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=403726925827 2026-02-19T16:28:35.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 403726925827' 2026-02-19T16:28:35.222 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 403726925827 2026-02-19T16:28:35.222 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:35.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 403726925827 -lt 403726925827 2026-02-19T16:28:35.451 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:35.451 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411355 2026-02-19T16:28:35.452 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:35.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:35.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411355 2026-02-19T16:28:35.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:35.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411355 2026-02-19T16:28:35.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411355' 2026-02-19T16:28:35.454 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411355 2026-02-19T16:28:35.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:28:35.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411355 -lt 77309411355 2026-02-19T16:28:35.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:35.687 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-360777252870 2026-02-19T16:28:35.687 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:35.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:28:35.688 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-360777252870 2026-02-19T16:28:35.688 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:35.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=360777252870 2026-02-19T16:28:35.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 360777252870' 2026-02-19T16:28:35.690 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 360777252870 2026-02-19T16:28:35.690 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:35.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 360777252870 -lt 360777252870 2026-02-19T16:28:35.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:28:35.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:35.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:36.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:28:36.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:28:36.232 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:28:36.232 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:28:36.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:28:36.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:28:36.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:28:36.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:28:36.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:28:36.457 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:28:36.458 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:36.458 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:36.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:28:36.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:28:36.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:28:36.773 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T16:28:36.775 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:28:36.775 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:28:36.775 INFO:tasks.workunit.client.0.trial127.stderr:256 bytes copied, 2.0648e-05 s, 12.4 MB/s 2026-02-19T16:28:36.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:36.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:36.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:36.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:36.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:36.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:36.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:36.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:28:36.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:28:36.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:28:36.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:28:36.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:28:36.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:28:37.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:28:37.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:37.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:28:37.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:28:37.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:28:37.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:28:37.020 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:28:37.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:37.020 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:28:37.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:28:37.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:28:37.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:28:37.020 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:28:37.033 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:37.030+0000 7f69aaa36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:37.034 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:37.031+0000 7f69aaa36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:37.035 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:37.032+0000 7f69aaa36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:37.174 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:37.171+0000 7f69aaa36780 -1 Falling back to public interface 2026-02-19T16:28:37.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:28:37.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:37.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:28:37.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:37.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:37.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:37.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:37.246 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:28:37.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:37.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:37.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:37.513 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:37.511+0000 7f69aaa36780 -1 osd.1 95 log_to_monitors true 2026-02-19T16:28:38.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:38.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:38.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:38.469 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:38.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:38.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:38.699 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 99 up_thru 99 down_at 96 last_clean_interval [94,95) [v2:127.0.0.1:6810/2523859421,v1:127.0.0.1:6811/2523859421] [v2:127.0.0.1:6812/2523859421,v1:127.0.0.1:6813/2523859421] exists,up 2d6c1039-a418-4c84-b873-ae0882692e74 2026-02-19T16:28:38.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:28:38.700 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:28:38.701 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:28:38.701 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:28:38.701 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:28:38.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:28:38.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:28:38.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:28:38.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:28:38.789 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:28:38.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:28:39.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:28:39.014 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:39.015 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:28:39.015 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:28:39.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:28:39.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:39.015 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:28:39.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-19T16:28:39.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-19T16:28:39.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807' 2026-02-19T16:28:39.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:39.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:28:39.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=425201762307 2026-02-19T16:28:39.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 425201762307 2026-02-19T16:28:39.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-425201762307' 2026-02-19T16:28:39.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:39.146 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:28:39.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411358 2026-02-19T16:28:39.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411358 2026-02-19T16:28:39.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-425201762307 2-77309411358' 2026-02-19T16:28:39.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:39.208 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:28:39.271 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=360777252873 2026-02-19T16:28:39.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 360777252873 2026-02-19T16:28:39.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-425201762307 2-77309411358 3-360777252873' 2026-02-19T16:28:39.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:39.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803807 2026-02-19T16:28:39.272 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:39.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:28:39.273 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-19T16:28:39.273 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:39.274 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803807 2026-02-19T16:28:39.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-19T16:28:39.275 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803807 2026-02-19T16:28:39.275 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:39.499 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-19T16:28:39.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:40.500 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:28:40.501 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:40.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-19T16:28:40.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:41.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:28:41.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:41.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-19T16:28:41.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:41.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-425201762307 2026-02-19T16:28:41.960 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:41.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:41.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-425201762307 2026-02-19T16:28:41.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:41.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=425201762307 2026-02-19T16:28:41.962 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 425201762307' 2026-02-19T16:28:41.962 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 425201762307 2026-02-19T16:28:41.963 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:42.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 425201762307 -lt 425201762307 2026-02-19T16:28:42.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:42.191 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411358 2026-02-19T16:28:42.191 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:42.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:42.192 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411358 2026-02-19T16:28:42.192 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:42.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411358 2026-02-19T16:28:42.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411358' 2026-02-19T16:28:42.194 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411358 2026-02-19T16:28:42.194 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:28:42.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411358 -lt 77309411358 2026-02-19T16:28:42.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:42.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-360777252873 2026-02-19T16:28:42.421 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:42.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:28:42.423 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-360777252873 2026-02-19T16:28:42.423 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:42.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=360777252873 2026-02-19T16:28:42.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 360777252873' 2026-02-19T16:28:42.424 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 360777252873 2026-02-19T16:28:42.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:28:42.653 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 360777252873 -lt 360777252873 2026-02-19T16:28:42.654 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:28:42.654 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:42.654 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:28:42.981 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:28:43.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:28:43.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:28:43.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:28:43.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:28:43.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:28:43.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:28:43.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:28:43.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:28:43.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:28:43.844 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-0-256 fail 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-0-256 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T16:28:43.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-95545-0-256 td/test-erasure-eio/COPY 2026-02-19T16:28:43.873 INFO:tasks.workunit.client.0.trial127.stderr:error getting pool-jerasure/obj-size-95545-0-256: (5) Input/output error 2026-02-19T16:28:43.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T16:28:43.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:28:43.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:28:43.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:28:43.875 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:28:44.108 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:28:44.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:28:44.393 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:28:44.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:28:44.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:28:44.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:28:44.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:28:44.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:44.402 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:44.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:44.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:44.403 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:44.511 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:44.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:28:44.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:28:44.513 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:28:44.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:28:44.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:28:44.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:28:44.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:28:44.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:28:44.517 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:28:44.518 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:28:44.518 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:28:44.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:28:44.519 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:28:44.535 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:28:44.536 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:44.536 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:44.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:28:44.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:28:44.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:28:44.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:28:44.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:28:44.539 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:28:44.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:28:44.540 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:28:44.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:28:44.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:28:44.542 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:28:44.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:28:44.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:28:44.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:28:44.543 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:28:44.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:28:44.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:28:44.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:28:44.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:28:44.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:44.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:44.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:28:44.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:28:44.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:28:44.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:28:44.548 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:28:44.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:44.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:44.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:28:44.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:28:44.551 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:28:44.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:28:44.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:28:44.569 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:28:44.570 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:44.570 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:44.570 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:44.570 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:44.570 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:44.570 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:44.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:28:44.593 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:28:44.596 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:28:44.597 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:44.597 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:44.597 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:28:44.597 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:28:44.597 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:28:44.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:28:44.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:28:44.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:28:44.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:28:44.639 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:28:44.640 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:28:44.640 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:28:44.641 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:28:44.641 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:28:44.641 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:44.641 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:44.641 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:28:44.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:28:44.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:28:44.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:28:44.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:28:44.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:28:44.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:28:44.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:28:44.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:28:44.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:28:44.782 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:28:44.782 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:44.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:44.783 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:44.783 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:44.783 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:44.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:44.784 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:28:44.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:28:44.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:28:44.799 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:28:44.894 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:28:44.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:28:45.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:28:45.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:28:45.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:28:45.904 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:28:45.904 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:45.904 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:45.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:28:45.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:28:45.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:28:45.943 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:28:45.947 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:28:45.949 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:28:44.585+0000 7ff5c0e2cc00 0 load: jerasure load: lrc 2026-02-19T16:28:45.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-19T16:28:45.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-19T16:28:45.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-19T16:28:45.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T16:28:45.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:28:45.950 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T16:28:45.951 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T16:28:45.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:28:45.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:28:45.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:45.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:28:45.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e1843d61-9cb7-4037-98c8-14df76e2c7ec 2026-02-19T16:28:45.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e1843d61-9cb7-4037-98c8-14df76e2c7ec' 2026-02-19T16:28:45.959 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 e1843d61-9cb7-4037-98c8-14df76e2c7ec 2026-02-19T16:28:45.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:28:45.965 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA9OpdpZjpzORAAWXbEY5hnAsmIWZZ2VdtQ4Q== 2026-02-19T16:28:45.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA9OpdpZjpzORAAWXbEY5hnAsmIWZZ2VdtQ4Q=="}' 2026-02-19T16:28:45.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e1843d61-9cb7-4037-98c8-14df76e2c7ec -i td/test-erasure-eio/0/new.json 2026-02-19T16:28:46.068 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:28:46.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:28:46.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA9OpdpZjpzORAAWXbEY5hnAsmIWZZ2VdtQ4Q== --osd-uuid e1843d61-9cb7-4037-98c8-14df76e2c7ec 2026-02-19T16:28:46.091 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.089+0000 7f6d08a0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:46.092 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.091+0000 7f6d08a0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:46.093 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.091+0000 7f6d08a0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:46.094 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.092+0000 7f6d08a0e780 -1 bdev(0x298cc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:28:46.094 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.092+0000 7f6d08a0e780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:28:46.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:28:46.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:28:46.549 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:28:46.549 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:28:46.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:28:46.652 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:28:46.653 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:28:46.653 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:46.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:28:46.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:28:46.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:28:46.668 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.666+0000 7fe83b41b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:46.669 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.667+0000 7fe83b41b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:46.670 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.668+0000 7fe83b41b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:46.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:28:46.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:46.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:28:46.752 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:46.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:46.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:46.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:46.753 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:28:46.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:46.753 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:28:46.849 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:46.875 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:46.874+0000 7fe83b41b780 -1 Falling back to public interface 2026-02-19T16:28:47.146 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:47.144+0000 7fe83b41b780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:28:47.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:47.850 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:47.850 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:28:47.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:47.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:47.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:28:48.073 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3669606488,v1:127.0.0.1:6803/3669606488] [v2:127.0.0.1:6804/3669606488,v1:127.0.0.1:6805/3669606488] exists,up e1843d61-9cb7-4037-98c8-14df76e2c7ec 2026-02-19T16:28:48.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:48.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:48.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:48.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:28:48.076 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:28:48.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:28:48.077 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:28:48.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c896a946-9aa4-48b3-8207-81e6fbca056b' 2026-02-19T16:28:48.078 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:28:48.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBAOpdpJoQlBRAA/2ij3q1WaQczK7DbWE03CA== 2026-02-19T16:28:48.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBAOpdpJoQlBRAA/2ij3q1WaQczK7DbWE03CA=="}' 2026-02-19T16:28:48.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c896a946-9aa4-48b3-8207-81e6fbca056b -i td/test-erasure-eio/1/new.json 2026-02-19T16:28:48.324 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:28:48.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:28:48.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAOpdpJoQlBRAA/2ij3q1WaQczK7DbWE03CA== --osd-uuid c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:28:48.348 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.346+0000 7f21ba685780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:48.349 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.348+0000 7f21ba685780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:48.350 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.348+0000 7f21ba685780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:48.350 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.349+0000 7f21ba685780 -1 bdev(0x2f8cc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:28:48.350 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.349+0000 7f21ba685780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:28:48.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:28:48.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:28:48.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:28:48.663 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:28:48.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:28:48.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:28:48.979 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:28:48.980 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:48.980 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:28:48.980 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:28:48.980 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:28:48.992 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.991+0000 7f9bc122d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:48.994 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.992+0000 7f9bc122d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:48.995 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:48.993+0000 7f9bc122d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:49.116 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:49.114+0000 7f9bc122d780 -1 Falling back to public interface 2026-02-19T16:28:49.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:28:49.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:49.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:28:49.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:49.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:49.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:49.206 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:49.207 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:28:49.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:49.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:49.435 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:49.466 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:49.464+0000 7f9bc122d780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:28:50.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:50.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:50.436 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:28:50.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:50.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:50.437 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:28:50.661 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/50267911,v1:127.0.0.1:6811/50267911] [v2:127.0.0.1:6812/50267911,v1:127.0.0.1:6813/50267911] exists,up c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:28:50.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:28:50.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:28:50.667 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:50.668 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1c2e0981-7db5-4e85-bef4-03c23113dc94' 2026-02-19T16:28:50.669 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:28:50.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:28:50.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBCOpdpgTYvKBAAeZ3kpGNOUE6cnH2svR2/7w== 2026-02-19T16:28:50.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBCOpdpgTYvKBAAeZ3kpGNOUE6cnH2svR2/7w=="}' 2026-02-19T16:28:50.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1c2e0981-7db5-4e85-bef4-03c23113dc94 -i td/test-erasure-eio/2/new.json 2026-02-19T16:28:50.912 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:28:50.920 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:28:50.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBCOpdpgTYvKBAAeZ3kpGNOUE6cnH2svR2/7w== --osd-uuid 1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:28:50.937 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:50.935+0000 7fa38b621780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:50.938 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:50.936+0000 7fa38b621780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:50.939 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:50.937+0000 7fa38b621780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:50.940 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:50.937+0000 7fa38b621780 -1 bdev(0x441ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:28:50.940 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:50.937+0000 7fa38b621780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:28:51.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:28:51.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:28:51.260 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:28:51.261 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:28:51.261 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:28:51.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:28:51.597 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:28:51.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:51.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:28:51.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:28:51.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:28:51.613 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:51.611+0000 7f75c302c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:51.614 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:51.612+0000 7f75c302c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:51.615 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:51.613+0000 7f75c302c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:51.736 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:51.734+0000 7f75c302c780 -1 Falling back to public interface 2026-02-19T16:28:51.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:28:51.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:51.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:28:51.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:51.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:51.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:51.826 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:28:51.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:51.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:51.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:28:52.044 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:52.042+0000 7f75c302c780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:28:52.053 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:53.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:53.054 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:53.054 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:28:53.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:53.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:53.055 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:28:53.283 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2025501754,v1:127.0.0.1:6819/2025501754] [v2:127.0.0.1:6820/2025501754,v1:127.0.0.1:6821/2025501754] exists,up 1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:28:53.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:53.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:53.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:28:53.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:28:53.289 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:28:53.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=94ab2e8b-0a61-4c3a-b083-aee78c794321 2026-02-19T16:28:53.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 94ab2e8b-0a61-4c3a-b083-aee78c794321' 2026-02-19T16:28:53.292 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 94ab2e8b-0a61-4c3a-b083-aee78c794321 2026-02-19T16:28:53.292 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:28:53.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBFOpdprJmYERAA43/VKsst4FHrrmQ5jFBxQQ== 2026-02-19T16:28:53.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBFOpdprJmYERAA43/VKsst4FHrrmQ5jFBxQQ=="}' 2026-02-19T16:28:53.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 94ab2e8b-0a61-4c3a-b083-aee78c794321 -i td/test-erasure-eio/3/new.json 2026-02-19T16:28:53.527 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:28:53.534 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:28:53.535 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBFOpdprJmYERAA43/VKsst4FHrrmQ5jFBxQQ== --osd-uuid 94ab2e8b-0a61-4c3a-b083-aee78c794321 2026-02-19T16:28:53.550 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:53.548+0000 7f2c90a37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:53.551 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:53.549+0000 7f2c90a37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:53.552 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:53.550+0000 7f2c90a37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:53.552 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:53.550+0000 7f2c90a37780 -1 bdev(0x27b8c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:28:53.552 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:53.551+0000 7f2c90a37780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:28:53.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:28:53.890 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:28:53.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:28:53.891 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:28:53.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:28:54.216 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:28:54.216 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:28:54.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:28:54.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:28:54.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:28:54.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:28:54.230 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:54.228+0000 7f98efa68780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:54.231 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:54.229+0000 7f98efa68780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:54.232 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:54.230+0000 7f98efa68780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:28:54.364 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:54.363+0000 7f98efa68780 -1 Falling back to public interface 2026-02-19T16:28:54.445 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:28:54.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:28:54.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:28:54.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:28:54.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:28:54.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:54.446 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:28:54.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:28:54.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:54.447 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:54.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:28:54.703 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:28:54.701+0000 7f98efa68780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:28:55.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:28:55.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:28:55.674 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:28:55.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:28:55.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:28:55.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:28:55.903 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4270112054,v1:127.0.0.1:6827/4270112054] [v2:127.0.0.1:6828/4270112054,v1:127.0.0.1:6829/4270112054] exists,up 94ab2e8b-0a61-4c3a-b083-aee78c794321 2026-02-19T16:28:55.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:28:55.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:28:55.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:28:55.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:28:55.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:28:55.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:28:55.904 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:28:55.904 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:28:55.904 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:28:55.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:28:55.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:28:55.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:28:55.949 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:28:55.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:28:55.955 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:28:46.876+0000 7fe83b41b780 0 load: jerasure load: lrc 2026-02-19T16:28:55.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-19T16:28:55.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T16:28:55.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:28:55.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:28:55.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T16:28:55.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:28:55.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T16:28:55.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:28:55.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T16:28:56.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:28:56.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:28:56.577 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:28:56.584 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:28:57.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:28:57.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:28:57.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:28:57.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:28:57.587 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:28:57.588 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:28:57.588 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:28:57.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:28:57.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:28:57.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:28:57.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:28:57.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:28:57.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:28:57.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:28:57.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:28:57.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:28:57.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:28:57.903 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:28:57.903 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:28:57.903 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:28:57.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:28:57.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:57.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:28:57.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T16:28:57.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T16:28:57.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T16:28:57.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:57.968 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:28:58.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-19T16:28:58.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-19T16:28:58.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-19T16:28:58.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:58.033 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:28:58.099 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T16:28:58.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T16:28:58.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411331' 2026-02-19T16:28:58.100 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:28:58.100 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:28:58.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T16:28:58.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T16:28:58.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411331 3-111669149699' 2026-02-19T16:28:58.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:58.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-19T16:28:58.165 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:58.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:28:58.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T16:28:58.166 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:58.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-19T16:28:58.168 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T16:28:58.168 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803781 2026-02-19T16:28:58.168 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:58.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-19T16:28:58.395 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:28:59.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:28:59.396 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:28:59.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-19T16:28:59.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:59.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-19T16:28:59.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:59.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:28:59.628 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-19T16:28:59.628 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:59.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-19T16:28:59.629 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 55834574853 2026-02-19T16:28:59.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-19T16:28:59.630 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:28:59.861 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-19T16:28:59.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:28:59.862 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-19T16:28:59.862 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:28:59.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:28:59.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T16:28:59.863 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:28:59.864 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-19T16:28:59.865 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T16:28:59.865 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411331 2026-02-19T16:28:59.865 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:00.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T16:29:00.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:00.096 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-19T16:29:00.096 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:00.096 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:00.097 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T16:29:00.097 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:00.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-19T16:29:00.098 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T16:29:00.099 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149699 2026-02-19T16:29:00.099 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:00.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-19T16:29:00.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:00.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:00.333 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:00.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:00.654 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:00.655 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:00.655 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:00.655 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:00.655 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:00.655 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:00.655 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:00.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:00.876 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:00.876 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:00.876 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-95545-1-10 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-95545-1-10 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-95545-1-10 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:29:01.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:01.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:29:01.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:01.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:29:01.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:01.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:29:01.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-95545-1-10 td/test-erasure-eio/ORIGINAL 2026-02-19T16:29:01.208 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-95545-1-10 td/test-erasure-eio 1 10 set 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-1-10 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-1-10 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-1-10 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-1-10 2026-02-19T16:29:01.209 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:29:01.441 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:29:01.441 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:01.441 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:29:01.441 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:29:01.442 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:29:01.442 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:29:01.442 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T16:29:01.442 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:29:01.723 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:29:01.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:29:01.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T16:29:01.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T16:29:01.734 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:29:01.735 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:29:01.735 INFO:tasks.workunit.client.0.trial127.stderr:10 bytes copied, 3.5777e-05 s, 280 kB/s 2026-02-19T16:29:01.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:01.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:01.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:01.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:29:01.736 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:01.737 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:01.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:01.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:01.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:02.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:02.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:02.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:02.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:02.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:29:02.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:02.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:02.041 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:02.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:29:02.188 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:29:02.189 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:02.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:02.191 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:29:02.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:29:02.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:02.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:02.193 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:02.202 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:02.200+0000 7fa699410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:02.204 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:02.202+0000 7fa699410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:02.204 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:02.203+0000 7fa699410780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:02.365 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:02.363+0000 7fa699410780 -1 Falling back to public interface 2026-02-19T16:29:02.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:29:02.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:02.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:29:02.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:02.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:02.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:02.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:02.415 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:02.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:02.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:02.638 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:02.744 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:02.742+0000 7fa699410780 -1 osd.1 36 log_to_monitors true 2026-02-19T16:29:03.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:03.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:03.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:03.641 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:03.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:03.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [13,36) [v2:127.0.0.1:6810/2019686443,v1:127.0.0.1:6811/2019686443] [v2:127.0.0.1:6812/2019686443,v1:127.0.0.1:6813/2019686443] exists,up c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:03.870 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:03.871 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:03.871 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:03.871 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:03.871 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:03.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:03.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:03.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:03.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:03.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:03.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:03.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:04.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:04.181 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:04.181 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:04.181 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:04.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:04.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:04.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:04.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T16:29:04.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T16:29:04.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T16:29:04.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:04.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:04.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659139 2026-02-19T16:29:04.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659139 2026-02-19T16:29:04.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-176093659139' 2026-02-19T16:29:04.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:04.313 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:04.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-19T16:29:04.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-19T16:29:04.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-176093659139 2-77309411335' 2026-02-19T16:29:04.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:04.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:04.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T16:29:04.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T16:29:04.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-176093659139 2-77309411335 3-111669149702' 2026-02-19T16:29:04.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:04.441 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-19T16:29:04.441 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:04.441 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:04.442 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T16:29:04.442 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:04.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-19T16:29:04.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T16:29:04.443 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803785 2026-02-19T16:29:04.444 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:04.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803785 2026-02-19T16:29:04.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:05.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:29:05.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:05.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T16:29:05.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:05.901 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-176093659139 2026-02-19T16:29:05.901 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:05.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:05.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-176093659139 2026-02-19T16:29:05.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:05.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659139 2026-02-19T16:29:05.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 176093659139' 2026-02-19T16:29:05.903 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 176093659139 2026-02-19T16:29:05.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:06.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-19T16:29:06.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:06.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411335 2026-02-19T16:29:06.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:06.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:06.137 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-19T16:29:06.137 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:06.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411335 2026-02-19T16:29:06.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-19T16:29:06.139 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 77309411335 2026-02-19T16:29:06.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:06.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-19T16:29:06.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:06.374 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-19T16:29:06.375 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:06.375 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:06.376 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T16:29:06.376 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:06.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-19T16:29:06.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T16:29:06.377 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149702 2026-02-19T16:29:06.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:06.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-19T16:29:06.604 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:06.605 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:06.605 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:06.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:06.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:06.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:06.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:06.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:06.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:06.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:06.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:07.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:07.148 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:07.148 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:07.148 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:07.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:07.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:07.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:07.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:29:07.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:29:07.780 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:29:07.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-1-10 2026-02-19T16:29:07.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:29:07.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:29:07.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-1-10 2026-02-19T16:29:07.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:29:07.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:29:07.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-1-10 td/test-erasure-eio/COPY 2026-02-19T16:29:07.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:29:07.809 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:29:07.810 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-95545-1-10 td/test-erasure-eio 2 10 set 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-1-10 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:29:07.811 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:29:07.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-1-10 2026-02-19T16:29:07.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:29:07.812 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-1-10 2026-02-19T16:29:07.812 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-1-10 2026-02-19T16:29:07.812 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:29:08.041 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:29:08.041 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:08.041 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:29:08.041 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:29:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:29:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:29:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-19T16:29:08.042 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:29:08.357 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:29:08.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:29:08.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T16:29:08.367 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T16:29:08.369 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:29:08.369 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:29:08.369 INFO:tasks.workunit.client.0.trial127.stderr:10 bytes copied, 4.6477e-05 s, 215 kB/s 2026-02-19T16:29:08.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:08.370 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:08.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:08.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:08.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:08.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:08.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:08.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:08.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:08.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T16:29:08.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:08.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:08.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-95545-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:08.820 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:08.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:08.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:start osd.2 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:08.823 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:29:08.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:29:08.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:08.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:08.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:08.834 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:08.832+0000 7f3517c0a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:08.836 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:08.833+0000 7f3517c0a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:08.837 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:08.834+0000 7f3517c0a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:08.963 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:08.961+0000 7f3517c0a780 -1 Falling back to public interface 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:09.045 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:09.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:09.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:09.267 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:09.265+0000 7f3517c0a780 -1 osd.2 45 log_to_monitors true 2026-02-19T16:29:09.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:10.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:10.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:10.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:10.277 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:10.277 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:10.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:10.512 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 up in weight 1 up_from 51 up_thru 38 down_at 48 last_clean_interval [18,45) [v2:127.0.0.1:6818/1491184985,v1:127.0.0.1:6819/1491184985] [v2:127.0.0.1:6820/1491184985,v1:127.0.0.1:6821/1491184985] exists,up 1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:10.513 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:10.514 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:10.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:10.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:10.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:10.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:10.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:10.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:10.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:10.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:10.599 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:10.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:10.828 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:10.828 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:10.828 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:10.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:10.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:10.828 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:10.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-19T16:29:10.893 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-19T16:29:10.893 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788' 2026-02-19T16:29:10.893 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:10.893 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:10.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659142 2026-02-19T16:29:10.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659142 2026-02-19T16:29:10.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-176093659142' 2026-02-19T16:29:10.958 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:10.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:11.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332099 2026-02-19T16:29:11.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332099 2026-02-19T16:29:11.022 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-176093659142 2-219043332099' 2026-02-19T16:29:11.023 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:11.023 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:11.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149706 2026-02-19T16:29:11.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149706 2026-02-19T16:29:11.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-176093659142 2-219043332099 3-111669149706' 2026-02-19T16:29:11.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:11.086 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803788 2026-02-19T16:29:11.086 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:11.087 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:11.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-19T16:29:11.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:11.088 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803788 2026-02-19T16:29:11.089 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-19T16:29:11.089 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803788 2026-02-19T16:29:11.089 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:11.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-19T16:29:11.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:12.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:29:12.315 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:12.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-19T16:29:12.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:13.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:29:13.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:13.772 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-19T16:29:13.772 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:13.772 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-176093659142 2026-02-19T16:29:13.773 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:13.774 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:13.774 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-176093659142 2026-02-19T16:29:13.774 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:13.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659142 2026-02-19T16:29:13.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 176093659142' 2026-02-19T16:29:13.776 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 176093659142 2026-02-19T16:29:13.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:14.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659143 -lt 176093659142 2026-02-19T16:29:14.002 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:14.002 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-219043332099 2026-02-19T16:29:14.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:14.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:14.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-219043332099 2026-02-19T16:29:14.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:14.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332099 2026-02-19T16:29:14.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 219043332099' 2026-02-19T16:29:14.005 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 219043332099 2026-02-19T16:29:14.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:14.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332099 -lt 219043332099 2026-02-19T16:29:14.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:14.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149706 2026-02-19T16:29:14.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:14.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:14.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149706 2026-02-19T16:29:14.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:14.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149706 2026-02-19T16:29:14.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149706' 2026-02-19T16:29:14.236 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149706 2026-02-19T16:29:14.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:14.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149706 -lt 111669149706 2026-02-19T16:29:14.471 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:14.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:14.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:14.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:14.790 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:14.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:14.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:14.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:14.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:14.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:14.791 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:15.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:15.016 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:15.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:15.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:15.338 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:15.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:15.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:15.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:29:15.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:29:15.655 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:29:15.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T16:29:15.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-1-10 2026-02-19T16:29:15.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:29:15.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:29:15.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-1-10 2026-02-19T16:29:15.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:29:15.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:29:15.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-1-10 td/test-erasure-eio/COPY 2026-02-19T16:29:15.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:29:15.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:29:15.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-95545-1-0 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-95545-1-0 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-95545-1-0 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:15.684 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:29:15.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:15.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:29:15.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-95545-1-0 td/test-erasure-eio/ORIGINAL 2026-02-19T16:29:15.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-95545-1-0 td/test-erasure-eio 1 0 set 2026-02-19T16:29:15.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-1-0 2026-02-19T16:29:15.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:29:15.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:29:15.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:29:15.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T16:29:15.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-1-0 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-1-0 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-1-0 2026-02-19T16:29:15.703 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:29:15.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:29:15.937 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:15.937 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:29:15.937 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:29:15.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:29:15.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:29:15.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T16:29:15.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:29:16.252 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:29:16.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:29:16.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T16:29:16.264 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T16:29:16.266 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:16.267 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:29:16.268 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:16.268 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:16.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:16.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:16.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:16.572 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:16.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:16.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:29:16.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:29:16.715 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:29:16.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:16.715 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:29:16.715 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:29:16.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:16.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:16.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:16.727 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:16.724+0000 7f3f2746d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:16.728 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:16.726+0000 7f3f2746d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:16.729 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:16.727+0000 7f3f2746d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:16.871 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:16.869+0000 7f3f2746d780 -1 Falling back to public interface 2026-02-19T16:29:16.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:29:16.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:16.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:29:16.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:16.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:16.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:16.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:16.945 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:16.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:16.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:17.169 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:17.192 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:17.189+0000 7f3f2746d780 -1 osd.1 56 log_to_monitors true 2026-02-19T16:29:18.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:18.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:18.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:18.170 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:18.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:18.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:18.398 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 60 up_thru 60 down_at 57 last_clean_interval [41,56) [v2:127.0.0.1:6810/2113718217,v1:127.0.0.1:6811/2113718217] [v2:127.0.0.1:6812/2113718217,v1:127.0.0.1:6813/2113718217] exists,up c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:18.399 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:18.400 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:18.400 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:18.400 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:18.400 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:18.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:18.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:18.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:18.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:18.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:18.488 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:18.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:18.712 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:18.712 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:18.712 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:18.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:18.712 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:18.713 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:18.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803792 2026-02-19T16:29:18.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803792 2026-02-19T16:29:18.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792' 2026-02-19T16:29:18.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:18.777 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:18.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037763 2026-02-19T16:29:18.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037763 2026-02-19T16:29:18.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-257698037763' 2026-02-19T16:29:18.843 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:18.843 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:18.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332102 2026-02-19T16:29:18.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332102 2026-02-19T16:29:18.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-257698037763 2-219043332102' 2026-02-19T16:29:18.906 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:18.906 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:18.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149709 2026-02-19T16:29:18.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149709 2026-02-19T16:29:18.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-257698037763 2-219043332102 3-111669149709' 2026-02-19T16:29:18.972 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:18.972 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803792 2026-02-19T16:29:18.972 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:18.973 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:18.973 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803792 2026-02-19T16:29:18.974 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:18.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803792 2026-02-19T16:29:18.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-19T16:29:18.975 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803792 2026-02-19T16:29:18.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:19.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-19T16:29:19.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:19.217 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-257698037763 2026-02-19T16:29:19.218 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:19.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:19.219 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-257698037763 2026-02-19T16:29:19.220 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:19.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=257698037763 2026-02-19T16:29:19.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 257698037763' 2026-02-19T16:29:19.221 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 257698037763 2026-02-19T16:29:19.222 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:19.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 257698037763 2026-02-19T16:29:19.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:20.455 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:29:20.455 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:20.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 257698037763 2026-02-19T16:29:20.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:21.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:29:21.683 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:21.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037763 -lt 257698037763 2026-02-19T16:29:21.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:21.915 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-219043332102 2026-02-19T16:29:21.915 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:21.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:21.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-219043332102 2026-02-19T16:29:21.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:21.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332102 2026-02-19T16:29:21.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 219043332102' 2026-02-19T16:29:21.917 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 219043332102 2026-02-19T16:29:21.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:22.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332103 -lt 219043332102 2026-02-19T16:29:22.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:22.146 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149709 2026-02-19T16:29:22.146 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:22.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:22.147 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149709 2026-02-19T16:29:22.148 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:22.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149709 2026-02-19T16:29:22.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149709' 2026-02-19T16:29:22.149 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149709 2026-02-19T16:29:22.149 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:22.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149709 -lt 111669149709 2026-02-19T16:29:22.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:22.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:22.377 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:22.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:22.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:22.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:22.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:22.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:22.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:22.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:22.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:22.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:22.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:22.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:22.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:23.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:23.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:23.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:23.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:29:23.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:29:23.561 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:29:23.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-1-0 2026-02-19T16:29:23.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:29:23.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:29:23.570 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-1-0 2026-02-19T16:29:23.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:29:23.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:29:23.571 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-1-0 td/test-erasure-eio/COPY 2026-02-19T16:29:23.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:29:23.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:29:23.592 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-19T16:29:23.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-19T16:29:23.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-95545-1-0 td/test-erasure-eio 2 0 set 2026-02-19T16:29:23.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-1-0 2026-02-19T16:29:23.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-1-0 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-1-0 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-1-0 2026-02-19T16:29:23.595 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:29:23.820 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:29:23.821 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:23.821 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:29:23.821 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:29:23.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:29:23.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:29:23.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-19T16:29:23.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:29:24.136 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:29:24.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T16:29:24.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T16:29:24.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:24.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:24.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:24.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:24.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:24.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:24.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T16:29:24.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:24.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:24.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-95545-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:24.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T16:29:24.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:24.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:24.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:29:24.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:24.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:24.588 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:24.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:24.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:24.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:24.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:29:24.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:29:24.590 INFO:tasks.workunit.client.0.trial127.stderr:start osd.2 2026-02-19T16:29:24.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:24.590 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:29:24.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:29:24.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:24.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:24.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:24.601 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:24.599+0000 7f49aa210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:24.602 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:24.601+0000 7f49aa210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:24.603 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:24.602+0000 7f49aa210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:24.731 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:24.730+0000 7f49aa210780 -1 Falling back to public interface 2026-02-19T16:29:24.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:29:24.814 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:24.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:25.022 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:25.020+0000 7f49aa210780 -1 osd.2 63 log_to_monitors true 2026-02-19T16:29:25.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:26.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:26.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:26.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:26.045 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:26.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:26.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:26.275 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 up in weight 1 up_from 69 up_thru 57 down_at 66 last_clean_interval [51,63) [v2:127.0.0.1:6818/1535710284,v1:127.0.0.1:6819/1535710284] [v2:127.0.0.1:6820/1535710284,v1:127.0.0.1:6821/1535710284] exists,up 1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:29:26.275 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:26.276 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:26.277 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:26.277 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:26.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:26.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:26.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:26.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:26.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:26.362 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:26.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:26.591 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:26.591 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:26.591 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:26.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:26.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:26.592 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:26.656 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803795 2026-02-19T16:29:26.656 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803795 2026-02-19T16:29:26.656 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795' 2026-02-19T16:29:26.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:26.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:26.720 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037766 2026-02-19T16:29:26.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037766 2026-02-19T16:29:26.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-257698037766' 2026-02-19T16:29:26.721 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:26.721 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:26.784 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743427 2026-02-19T16:29:26.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743427 2026-02-19T16:29:26.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-257698037766 2-296352743427' 2026-02-19T16:29:26.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:26.785 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:26.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149713 2026-02-19T16:29:26.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149713 2026-02-19T16:29:26.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-257698037766 2-296352743427 3-111669149713' 2026-02-19T16:29:26.851 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:26.852 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803795 2026-02-19T16:29:26.852 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:26.852 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:26.853 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803795 2026-02-19T16:29:26.853 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:26.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803795 2026-02-19T16:29:26.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-19T16:29:26.854 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803795 2026-02-19T16:29:26.854 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:27.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803795 2026-02-19T16:29:27.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:27.082 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-257698037766 2026-02-19T16:29:27.082 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:27.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:27.083 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-257698037766 2026-02-19T16:29:27.083 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:27.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=257698037766 2026-02-19T16:29:27.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 257698037766' 2026-02-19T16:29:27.084 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 257698037766 2026-02-19T16:29:27.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:27.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037766 -lt 257698037766 2026-02-19T16:29:27.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:27.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-296352743427 2026-02-19T16:29:27.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:27.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:27.315 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743427 2026-02-19T16:29:27.316 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:27.316 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=296352743427 2026-02-19T16:29:27.317 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 296352743427' 2026-02-19T16:29:27.317 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 296352743427 2026-02-19T16:29:27.317 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:27.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 296352743427 -lt 296352743427 2026-02-19T16:29:27.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:27.543 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149713 2026-02-19T16:29:27.543 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:27.544 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:27.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149713 2026-02-19T16:29:27.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:27.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149713 2026-02-19T16:29:27.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149713' 2026-02-19T16:29:27.546 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149713 2026-02-19T16:29:27.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:27.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149711 -lt 111669149713 2026-02-19T16:29:27.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:28.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:29:28.776 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:29.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149713 -lt 111669149713 2026-02-19T16:29:29.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:29.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:29.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:29.321 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:29.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:29.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:29.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:29.546 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:29.868 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:29.868 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:29.868 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:29.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:29:29.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:29:30.189 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-1-0 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-1-0 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:29:30.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-1-0 td/test-erasure-eio/COPY 2026-02-19T16:29:30.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:29:30.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:29:30.220 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:29:30.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-19T16:29:30.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T16:29:30.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T16:29:30.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-19T16:29:30.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T16:29:30.221 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-95545-1-256 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-95545-1-256 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-95545-1-256 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:29:30.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-95545-1-256 td/test-erasure-eio/ORIGINAL 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-95545-1-256 td/test-erasure-eio 1 256 add 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-1-256 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-1-256 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:29:30.240 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-1-256 2026-02-19T16:29:30.241 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-1-256 2026-02-19T16:29:30.241 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:29:30.479 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:29:30.479 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:30.479 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:29:30.480 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:29:30.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:29:30.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:29:30.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T16:29:30.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:29:30.758 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:30.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:31.074 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:31.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:31.160 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:29:31.161 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:31.162 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:29:31.162 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:29:31.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:31.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:31.163 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:31.176 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:31.175+0000 7fc2b6e44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:31.177 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:31.176+0000 7fc2b6e44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:31.178 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:31.177+0000 7fc2b6e44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:31.303 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:31.301+0000 7fc2b6e44780 -1 Falling back to public interface 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:31.390 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:31.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:31.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:31.616 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:31.614+0000 7fc2b6e44780 -1 osd.1 74 log_to_monitors true 2026-02-19T16:29:31.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:32.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:32.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:32.621 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:32.622 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:32.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:32.622 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:32.854 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [60,74) [v2:127.0.0.1:6810/4232500307,v1:127.0.0.1:6811/4232500307] [v2:127.0.0.1:6812/4232500307,v1:127.0.0.1:6813/4232500307] exists,up c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:29:32.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:32.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:32.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:32.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:32.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:32.854 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:32.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:32.855 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:32.855 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:32.855 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:32.855 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:32.855 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:32.855 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:32.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:32.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:32.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:32.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:32.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:32.942 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:33.170 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:33.170 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:33.171 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:33.171 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:33.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:33.171 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:33.171 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:33.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803798 2026-02-19T16:29:33.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803798 2026-02-19T16:29:33.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798' 2026-02-19T16:29:33.239 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:33.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:33.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449091 2026-02-19T16:29:33.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449091 2026-02-19T16:29:33.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-335007449091' 2026-02-19T16:29:33.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:33.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:33.371 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743430 2026-02-19T16:29:33.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743430 2026-02-19T16:29:33.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-335007449091 2-296352743430' 2026-02-19T16:29:33.372 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:33.372 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:33.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149716 2026-02-19T16:29:33.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149716 2026-02-19T16:29:33.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-335007449091 2-296352743430 3-111669149716' 2026-02-19T16:29:33.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:33.437 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803798 2026-02-19T16:29:33.437 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:33.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:33.439 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803798 2026-02-19T16:29:33.439 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:33.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803798 2026-02-19T16:29:33.440 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803798' 2026-02-19T16:29:33.440 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803798 2026-02-19T16:29:33.440 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:33.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803796 -lt 25769803798 2026-02-19T16:29:33.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:34.667 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:29:34.668 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:34.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803799 -lt 25769803798 2026-02-19T16:29:34.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:34.899 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-335007449091 2026-02-19T16:29:34.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:34.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:34.901 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-335007449091 2026-02-19T16:29:34.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:34.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=335007449091 2026-02-19T16:29:34.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 335007449091' 2026-02-19T16:29:34.903 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 335007449091 2026-02-19T16:29:34.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:35.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-19T16:29:35.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:35.133 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-296352743430 2026-02-19T16:29:35.133 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:35.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:35.134 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743430 2026-02-19T16:29:35.134 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:35.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=296352743430 2026-02-19T16:29:35.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 296352743430' 2026-02-19T16:29:35.135 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 296352743430 2026-02-19T16:29:35.135 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:35.363 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 296352743430 -lt 296352743430 2026-02-19T16:29:35.363 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:35.363 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149716 2026-02-19T16:29:35.364 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:35.364 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:35.365 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149716 2026-02-19T16:29:35.365 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:35.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149716 2026-02-19T16:29:35.366 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149716' 2026-02-19T16:29:35.366 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149716 2026-02-19T16:29:35.366 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:35.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149716 -lt 111669149716 2026-02-19T16:29:35.597 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:35.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:35.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:35.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:35.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:35.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:35.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:35.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:35.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:35.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:35.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:36.153 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:36.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:36.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:36.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:36.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:36.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:36.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:36.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T16:29:36.473 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:29:36.473 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:256 bytes copied, 3.2951e-05 s, 7.8 MB/s 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:36.474 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:36.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:36.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:36.475 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:36.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:36.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:36.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:36.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:36.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:29:36.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:36.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:36.581 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:36.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:36.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:29:36.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:29:36.731 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:29:36.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:36.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:29:36.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:29:36.732 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:36.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:36.733 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:36.747 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:36.745+0000 7f5cf840b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:36.748 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:36.747+0000 7f5cf840b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:36.749 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:36.748+0000 7f5cf840b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:36.884 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:36.882+0000 7f5cf840b780 -1 Falling back to public interface 2026-02-19T16:29:36.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:36.961 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:37.190 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:37.205 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:37.204+0000 7f5cf840b780 -1 osd.1 79 log_to_monitors true 2026-02-19T16:29:38.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:38.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:38.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:38.192 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:38.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:38.192 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:29:38.418 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [78,79) [v2:127.0.0.1:6810/2717193953,v1:127.0.0.1:6811/2717193953] [v2:127.0.0.1:6812/2717193953,v1:127.0.0.1:6813/2717193953] exists,up c896a946-9aa4-48b3-8207-81e6fbca056b 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:38.419 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:38.420 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:38.420 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:38.420 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:38.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:38.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:38.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:38.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:38.506 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:38.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:38.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:38.735 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:38.735 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:38.735 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:38.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:38.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:38.735 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:38.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803802 2026-02-19T16:29:38.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803802 2026-02-19T16:29:38.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802' 2026-02-19T16:29:38.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:38.801 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:38.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-19T16:29:38.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-19T16:29:38.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-356482285571' 2026-02-19T16:29:38.869 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:38.869 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:38.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743433 2026-02-19T16:29:38.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743433 2026-02-19T16:29:38.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-356482285571 2-296352743433' 2026-02-19T16:29:38.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:38.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:38.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149719 2026-02-19T16:29:38.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149719 2026-02-19T16:29:38.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-356482285571 2-296352743433 3-111669149719' 2026-02-19T16:29:38.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:38.999 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803802 2026-02-19T16:29:39.000 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:39.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:39.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803802 2026-02-19T16:29:39.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:39.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803802 2026-02-19T16:29:39.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-19T16:29:39.003 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803802 2026-02-19T16:29:39.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:39.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-19T16:29:39.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:39.231 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-356482285571 2026-02-19T16:29:39.231 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:39.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:39.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-356482285571 2026-02-19T16:29:39.233 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:39.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285571 2026-02-19T16:29:39.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 356482285571' 2026-02-19T16:29:39.234 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 356482285571 2026-02-19T16:29:39.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:39.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 356482285571 2026-02-19T16:29:39.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:40.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:29:40.462 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:40.688 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 356482285571 2026-02-19T16:29:40.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:41.689 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:29:41.690 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:41.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-19T16:29:41.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:41.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-296352743433 2026-02-19T16:29:41.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:41.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:41.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743433 2026-02-19T16:29:41.918 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:41.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=296352743433 2026-02-19T16:29:41.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 296352743433' 2026-02-19T16:29:41.920 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 296352743433 2026-02-19T16:29:41.920 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:42.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 296352743434 -lt 296352743433 2026-02-19T16:29:42.149 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:42.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149719 2026-02-19T16:29:42.150 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:42.151 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:42.151 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149719 2026-02-19T16:29:42.151 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:42.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149719 2026-02-19T16:29:42.152 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149719' 2026-02-19T16:29:42.152 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149719 2026-02-19T16:29:42.153 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:42.385 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149719 -lt 111669149719 2026-02-19T16:29:42.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:42.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:42.386 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:42.705 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:42.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:42.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:42.931 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:42.931 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:43.249 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:43.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:43.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:43.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:29:43.250 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:29:43.566 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:29:43.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-1-256 2026-02-19T16:29:43.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:29:43.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:29:43.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-1-256 2026-02-19T16:29:43.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:29:43.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:29:43.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-1-256 td/test-erasure-eio/COPY 2026-02-19T16:29:43.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:29:43.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:29:43.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-19T16:29:43.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-19T16:29:43.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-95545-1-256 td/test-erasure-eio 2 256 add 2026-02-19T16:29:43.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-95545-1-256 2026-02-19T16:29:43.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T16:29:43.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T16:29:43.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T16:29:43.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-95545-1-256 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-95545-1-256 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-95545-1-256 2026-02-19T16:29:43.600 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:29:43.831 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:29:43.831 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:43.832 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:29:43.832 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:29:43.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T16:29:43.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T16:29:43.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-19T16:29:43.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T16:29:44.147 INFO:tasks.workunit.client.0.trial127.stderr:noout is set 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:44.159 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T16:29:44.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:44.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:29:44.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:44.160 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:44.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:44.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:44.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:44.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-95545-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:29:44.554 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:44.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:start osd.2 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:44.556 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:29:44.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:29:44.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:44.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:44.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:44.570 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:44.567+0000 7f6fbec42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:44.571 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:44.569+0000 7f6fbec42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:44.572 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:44.570+0000 7f6fbec42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:44.705 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:44.703+0000 7f6fbec42780 -1 Falling back to public interface 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:44.785 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:45.013 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:45.020 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:45.018+0000 7f6fbec42780 -1 osd.2 85 log_to_monitors true 2026-02-19T16:29:46.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:46.014 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:46.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:46.015 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:46.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:46.015 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 up in weight 1 up_from 92 up_thru 80 down_at 89 last_clean_interval [69,85) [v2:127.0.0.1:6818/190830108,v1:127.0.0.1:6819/190830108] [v2:127.0.0.1:6820/190830108,v1:127.0.0.1:6821/190830108] exists,up 1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:46.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:46.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:46.245 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:46.245 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:46.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:46.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:46.245 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:46.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:46.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:46.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:46.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:46.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:46.335 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:46.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:46.558 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:46.558 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:46.558 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:46.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:46.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:46.558 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:46.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803805 2026-02-19T16:29:46.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803805 2026-02-19T16:29:46.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805' 2026-02-19T16:29:46.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:46.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:46.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285574 2026-02-19T16:29:46.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285574 2026-02-19T16:29:46.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-356482285574' 2026-02-19T16:29:46.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:46.692 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:46.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991235 2026-02-19T16:29:46.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991235 2026-02-19T16:29:46.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-356482285574 2-395136991235' 2026-02-19T16:29:46.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:46.759 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:46.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149723 2026-02-19T16:29:46.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149723 2026-02-19T16:29:46.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-356482285574 2-395136991235 3-111669149723' 2026-02-19T16:29:46.826 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:46.826 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803805 2026-02-19T16:29:46.826 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:46.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:46.827 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803805 2026-02-19T16:29:46.827 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:46.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803805 2026-02-19T16:29:46.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803805' 2026-02-19T16:29:46.829 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803805 2026-02-19T16:29:46.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:47.056 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803805 2026-02-19T16:29:47.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:47.057 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-356482285574 2026-02-19T16:29:47.057 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:47.058 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:47.059 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-356482285574 2026-02-19T16:29:47.059 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:47.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285574 2026-02-19T16:29:47.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 356482285574' 2026-02-19T16:29:47.060 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 356482285574 2026-02-19T16:29:47.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:47.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285574 -lt 356482285574 2026-02-19T16:29:47.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:47.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-395136991235 2026-02-19T16:29:47.288 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:47.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:47.289 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-395136991235 2026-02-19T16:29:47.289 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:47.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991235 2026-02-19T16:29:47.290 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 395136991235' 2026-02-19T16:29:47.290 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 395136991235 2026-02-19T16:29:47.291 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:47.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-19T16:29:47.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:47.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149723 2026-02-19T16:29:47.521 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:47.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:47.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149723 2026-02-19T16:29:47.522 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:47.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149723 2026-02-19T16:29:47.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149723' 2026-02-19T16:29:47.523 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149723 2026-02-19T16:29:47.524 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:47.750 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149723 -lt 111669149723 2026-02-19T16:29:47.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:47.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:47.751 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:48.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:48.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:48.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:48.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:48.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:48.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:48.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:48.069 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:48.299 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:48.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:48.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:48.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:48.616 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:48.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:48.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:48.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records in 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:1+0 records out 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:256 bytes copied, 2.2172e-05 s, 11.5 MB/s 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:48.619 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:48.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:48.724 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:48.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:48.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:29:48.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:29:48.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T16:29:48.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:29:48.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:48.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-95545-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T16:29:48.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T16:29:48.870 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:29:48.871 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:48.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:48.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:48.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:29:48.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:29:48.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr:start osd.2 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:29:48.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:29:48.886 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:48.884+0000 7fc409bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:48.888 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:48.885+0000 7fc409bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:48.889 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:48.886+0000 7fc409bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:49.012 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:49.010+0000 7fc409bfd780 -1 Falling back to public interface 2026-02-19T16:29:49.102 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:49.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:49.300 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:49.297+0000 7fc409bfd780 -1 osd.2 93 log_to_monitors true 2026-02-19T16:29:49.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:50.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:50.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:50.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:50.333 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:50.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:50.334 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 up in weight 1 up_from 97 up_thru 80 down_at 94 last_clean_interval [92,93) [v2:127.0.0.1:6818/3619351162,v1:127.0.0.1:6819/3619351162] [v2:127.0.0.1:6820/3619351162,v1:127.0.0.1:6821/3619351162] exists,up 1c2e0981-7db5-4e85-bef4-03c23113dc94 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:29:50.562 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:29:50.563 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:29:50.563 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:29:50.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:29:50.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:29:50.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:29:50.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:29:50.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:29:50.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:29:50.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:29:50.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:29:50.651 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:29:50.878 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:29:50.878 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:29:50.878 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:29:50.878 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:29:50.878 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:29:50.878 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:50.878 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:29:50.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803808 2026-02-19T16:29:50.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803808 2026-02-19T16:29:50.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808' 2026-02-19T16:29:50.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:50.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:29:51.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285577 2026-02-19T16:29:51.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285577 2026-02-19T16:29:51.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-356482285577' 2026-02-19T16:29:51.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:51.010 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:29:51.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=416611827715 2026-02-19T16:29:51.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 416611827715 2026-02-19T16:29:51.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-356482285577 2-416611827715' 2026-02-19T16:29:51.076 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:29:51.076 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:29:51.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149726 2026-02-19T16:29:51.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149726 2026-02-19T16:29:51.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-356482285577 2-416611827715 3-111669149726' 2026-02-19T16:29:51.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:51.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803808 2026-02-19T16:29:51.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:51.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:29:51.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803808 2026-02-19T16:29:51.143 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:51.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803808 2026-02-19T16:29:51.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803808' 2026-02-19T16:29:51.145 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803808 2026-02-19T16:29:51.145 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:51.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803806 -lt 25769803808 2026-02-19T16:29:51.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:52.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:29:52.371 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:52.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803806 -lt 25769803808 2026-02-19T16:29:52.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:29:53.600 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:29:53.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:29:53.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803808 -lt 25769803808 2026-02-19T16:29:53.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:53.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-356482285577 2026-02-19T16:29:53.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:53.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:29:53.830 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-356482285577 2026-02-19T16:29:53.830 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:53.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285577 2026-02-19T16:29:53.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 356482285577' 2026-02-19T16:29:53.832 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 356482285577 2026-02-19T16:29:53.832 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:29:54.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285578 -lt 356482285577 2026-02-19T16:29:54.063 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:54.063 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-416611827715 2026-02-19T16:29:54.064 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:54.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:29:54.065 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-416611827715 2026-02-19T16:29:54.065 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:54.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=416611827715 2026-02-19T16:29:54.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 416611827715' 2026-02-19T16:29:54.066 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 416611827715 2026-02-19T16:29:54.066 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:29:54.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-19T16:29:54.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:29:54.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149726 2026-02-19T16:29:54.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:29:54.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:29:54.297 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149726 2026-02-19T16:29:54.297 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:29:54.298 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149726 2026-02-19T16:29:54.298 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149726' 2026-02-19T16:29:54.298 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149726 2026-02-19T16:29:54.299 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:29:54.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149726 -lt 111669149726 2026-02-19T16:29:54.527 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:29:54.527 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:54.527 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:54.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:29:54.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:29:54.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:29:54.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:29:54.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:29:54.848 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:29:54.848 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:29:54.848 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:29:55.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:29:55.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:29:55.074 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:29:55.074 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:29:55.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:29:55.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:29:55.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:29:55.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T16:29:55.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T16:29:55.706 INFO:tasks.workunit.client.0.trial127.stderr:noout is unset 2026-02-19T16:29:55.716 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T16:29:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-95545-1-256 2026-02-19T16:29:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:29:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:29:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-95545-1-256 2026-02-19T16:29:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:29:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:29:55.717 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-95545-1-256 td/test-erasure-eio/COPY 2026-02-19T16:29:55.734 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:29:55.735 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:29:55.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:29:55.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:29:55.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:29:55.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:29:55.979 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:29:55.987 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:29:56.268 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:29:56.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:29:56.278 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:29:56.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:29:56.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:29:56.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:56.279 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:56.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:56.279 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:56.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:56.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:56.387 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:29:56.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:29:56.389 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:29:56.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:29:56.390 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:29:56.391 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:29:56.391 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:29:56.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:29:56.392 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:29:56.392 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:29:56.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:29:56.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:29:56.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:29:56.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:29:56.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:56.411 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:56.411 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:29:56.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:29:56.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:29:56.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:29:56.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:29:56.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:29:56.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:29:56.414 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:29:56.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:29:56.415 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:29:56.416 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:29:56.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:29:56.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:29:56.417 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:29:56.417 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:29:56.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:29:56.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:29:56.419 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:29:56.419 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:29:56.420 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:29:56.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:29:56.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:29:56.422 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:29:56.422 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:56.422 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:56.422 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:29:56.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:29:56.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:29:56.423 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:29:56.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:29:56.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:56.424 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:56.424 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:29:56.425 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:29:56.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:29:56.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:29:56.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:29:56.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:29:56.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:29:56.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:29:56.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:29:56.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:29:56.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:29:56.427 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:29:56.445 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:29:56.445 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:56.445 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:56.445 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:56.445 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:56.445 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:56.445 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:56.446 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:29:56.469 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:29:56.470 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:29:56.471 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:29:56.471 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:56.471 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:56.471 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:29:56.471 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:29:56.471 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:29:56.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:29:56.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:29:56.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:29:56.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:29:56.515 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:29:56.515 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:29:56.515 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:29:56.515 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:29:56.516 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:29:56.516 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:56.516 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:56.516 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:29:56.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:29:56.516 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:29:56.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:29:56.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:29:56.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:29:56.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:29:56.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:29:56.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:29:56.558 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:29:56.656 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:29:56.656 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:56.656 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:56.656 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:56.656 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:56.657 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:56.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:56.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:29:56.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:29:56.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:29:56.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:29:56.769 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:29:56.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:29:57.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:29:57.778 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:29:57.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:29:57.779 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:29:57.779 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:57.779 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:57.779 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:29:57.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:29:57.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:29:57.818 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:29:57.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:29:57.823 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:29:56.462+0000 7f8050646c00 0 load: jerasure load: lrc 2026-02-19T16:29:57.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-19T16:29:57.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-19T16:29:57.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-19T16:29:57.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T16:29:57.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:29:57.825 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T16:29:57.825 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:29:57.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:57.831 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:57.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=28eac685-5f62-4b0d-933d-e1a1522ac67f 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 28eac685-5f62-4b0d-933d-e1a1522ac67f' 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 28eac685-5f62-4b0d-933d-e1a1522ac67f 2026-02-19T16:29:57.833 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:29:57.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCFOpdpsTX9MRAA4zlVJOsozjdd00x+PNbhdw== 2026-02-19T16:29:57.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCFOpdpsTX9MRAA4zlVJOsozjdd00x+PNbhdw=="}' 2026-02-19T16:29:57.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 28eac685-5f62-4b0d-933d-e1a1522ac67f -i td/test-erasure-eio/0/new.json 2026-02-19T16:29:57.941 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:29:57.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:29:57.948 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCFOpdpsTX9MRAA4zlVJOsozjdd00x+PNbhdw== --osd-uuid 28eac685-5f62-4b0d-933d-e1a1522ac67f 2026-02-19T16:29:57.964 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:57.963+0000 7f876862c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:57.966 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:57.964+0000 7f876862c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:57.967 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:57.965+0000 7f876862c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:57.967 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:57.965+0000 7f876862c780 -1 bdev(0x3130c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:29:57.967 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:57.966+0000 7f876862c780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:29:58.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:29:58.396 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:29:58.397 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:29:58.398 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:29:58.398 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:29:58.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:29:58.504 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:29:58.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:29:58.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:29:58.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:29:58.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:29:58.518 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:58.517+0000 7f1a6add4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:58.520 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:58.518+0000 7f1a6add4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:58.520 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:58.519+0000 7f1a6add4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:29:58.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:29:58.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:29:58.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:29:58.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:29:58.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:29:58.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:58.606 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:29:58.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:29:58.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:58.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:29:58.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:29:58.757 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:58.755+0000 7f1a6add4780 -1 Falling back to public interface 2026-02-19T16:29:58.984 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:29:58.983+0000 7f1a6add4780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:29:59.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:29:59.703 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:29:59.703 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:29:59.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:29:59.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:29:59.704 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:29:59.928 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2901437181,v1:127.0.0.1:6803/2901437181] [v2:127.0.0.1:6804/2901437181,v1:127.0.0.1:6805/2901437181] exists,up 28eac685-5f62-4b0d-933d-e1a1522ac67f 2026-02-19T16:29:59.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:29:59.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:29:59.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:29:59.930 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:59.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:29:59.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=49d20c1c-367f-49cc-bf64-5deb5986d5df 2026-02-19T16:29:59.932 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 49d20c1c-367f-49cc-bf64-5deb5986d5df' 2026-02-19T16:29:59.933 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 49d20c1c-367f-49cc-bf64-5deb5986d5df 2026-02-19T16:29:59.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:29:59.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCHOpdpGBoUOBAA0alAQ2Rm/viofkLnLJJiSQ== 2026-02-19T16:29:59.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCHOpdpGBoUOBAA0alAQ2Rm/viofkLnLJJiSQ=="}' 2026-02-19T16:29:59.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 49d20c1c-367f-49cc-bf64-5deb5986d5df -i td/test-erasure-eio/1/new.json 2026-02-19T16:30:00.169 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:00.177 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:30:00.178 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCHOpdpGBoUOBAA0alAQ2Rm/viofkLnLJJiSQ== --osd-uuid 49d20c1c-367f-49cc-bf64-5deb5986d5df 2026-02-19T16:30:00.194 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.193+0000 7fd44aa3c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:00.196 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.194+0000 7fd44aa3c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:00.197 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.195+0000 7fd44aa3c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:00.197 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.195+0000 7fd44aa3c780 -1 bdev(0x3376c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:00.197 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.196+0000 7fd44aa3c780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:30:00.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:30:00.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:00.520 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:30:00.521 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:30:00.521 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:00.838 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:30:00.839 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:30:00.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:00.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:00.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:00.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:00.854 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.852+0000 7f0b0a067780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:00.855 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.853+0000 7f0b0a067780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:00.856 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.854+0000 7f0b0a067780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:00.993 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:00.991+0000 7f0b0a067780 -1 Falling back to public interface 2026-02-19T16:30:01.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:30:01.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:01.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:30:01.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:01.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:01.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:01.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:01.065 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:01.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:01.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:01.291 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:01.360 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:01.359+0000 7f0b0a067780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:30:02.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:02.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:02.293 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:02.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:02.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:02.293 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:02.521 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/16870107,v1:127.0.0.1:6811/16870107] [v2:127.0.0.1:6812/16870107,v1:127.0.0.1:6813/16870107] exists,up 49d20c1c-367f-49cc-bf64-5deb5986d5df 2026-02-19T16:30:02.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:02.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:02.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:02.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:02.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:30:02.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:02.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:30:02.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:02.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:02.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:30:02.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:02.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8a8729c7-6ccb-4029-90e0-2f741775ba80 2026-02-19T16:30:02.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 8a8729c7-6ccb-4029-90e0-2f741775ba80' 2026-02-19T16:30:02.530 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 8a8729c7-6ccb-4029-90e0-2f741775ba80 2026-02-19T16:30:02.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:02.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCKOpdpK1nYHxAAL1Bh/gTGp7n95iYMPqyUEQ== 2026-02-19T16:30:02.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCKOpdpK1nYHxAAL1Bh/gTGp7n95iYMPqyUEQ=="}' 2026-02-19T16:30:02.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8a8729c7-6ccb-4029-90e0-2f741775ba80 -i td/test-erasure-eio/2/new.json 2026-02-19T16:30:02.770 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:30:02.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:30:02.779 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCKOpdpK1nYHxAAL1Bh/gTGp7n95iYMPqyUEQ== --osd-uuid 8a8729c7-6ccb-4029-90e0-2f741775ba80 2026-02-19T16:30:02.795 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:02.793+0000 7fb16680a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:02.796 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:02.795+0000 7fb16680a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:02.797 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:02.796+0000 7fb16680a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:02.798 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:02.796+0000 7fb16680a780 -1 bdev(0x2d9ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:02.798 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:02.796+0000 7fb16680a780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:30:03.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:30:03.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:03.134 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:30:03.134 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:30:03.134 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:03.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:30:03.453 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:30:03.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:03.454 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:03.467 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:03.465+0000 7f76cbc5e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:03.469 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:03.467+0000 7f76cbc5e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:03.469 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:03.468+0000 7f76cbc5e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:03.598 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:03.596+0000 7f76cbc5e780 -1 Falling back to public interface 2026-02-19T16:30:03.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:30:03.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:03.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:30:03.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:03.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:03.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:03.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:03.681 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:03.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:03.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:03.889 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:03.887+0000 7f76cbc5e780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:30:03.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:04.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:04.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:04.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:04.910 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:04.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:04.910 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:05.134 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3316613742,v1:127.0.0.1:6819/3316613742] [v2:127.0.0.1:6820/3316613742,v1:127.0.0.1:6821/3316613742] exists,up 8a8729c7-6ccb-4029-90e0-2f741775ba80 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:30:05.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:05.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:30:05.137 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:05.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7e30a21-d2f2-49c7-a53a-2b12bba98a16 2026-02-19T16:30:05.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c7e30a21-d2f2-49c7-a53a-2b12bba98a16' 2026-02-19T16:30:05.138 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 c7e30a21-d2f2-49c7-a53a-2b12bba98a16 2026-02-19T16:30:05.138 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:05.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCNOpdp1ey+CBAA7gNud+3+MoA3mBJPp2e88A== 2026-02-19T16:30:05.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCNOpdp1ey+CBAA7gNud+3+MoA3mBJPp2e88A=="}' 2026-02-19T16:30:05.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7e30a21-d2f2-49c7-a53a-2b12bba98a16 -i td/test-erasure-eio/3/new.json 2026-02-19T16:30:05.379 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:30:05.386 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:30:05.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCNOpdp1ey+CBAA7gNud+3+MoA3mBJPp2e88A== --osd-uuid c7e30a21-d2f2-49c7-a53a-2b12bba98a16 2026-02-19T16:30:05.403 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:05.401+0000 7f2a3bdd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:05.404 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:05.402+0000 7f2a3bdd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:05.405 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:05.403+0000 7f2a3bdd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:05.405 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:05.404+0000 7f2a3bdd8780 -1 bdev(0x3c74c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:05.405 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:05.404+0000 7f2a3bdd8780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:30:05.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:30:05.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:05.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:30:05.731 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:30:05.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:06.050 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:30:06.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:30:06.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:06.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:06.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:06.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:06.065 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:06.064+0000 7fa34182e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:06.067 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:06.065+0000 7fa34182e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:06.068 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:06.066+0000 7fa34182e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:06.255 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:06.254+0000 7fa34182e780 -1 Falling back to public interface 2026-02-19T16:30:06.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:30:06.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:06.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:30:06.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:06.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:06.280 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:06.281 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:06.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:06.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:06.281 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:30:06.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:06.631 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:06.629+0000 7fa34182e780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:30:07.514 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:07.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:07.515 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:07.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:07.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:07.515 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:30:07.742 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1429392573,v1:127.0.0.1:6827/1429392573] [v2:127.0.0.1:6828/1429392573,v1:127.0.0.1:6829/1429392573] exists,up c7e30a21-d2f2-49c7-a53a-2b12bba98a16 2026-02-19T16:30:07.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:07.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:07.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:07.742 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:30:07.742 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:30:07.743 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:30:07.743 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:07.743 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:07.743 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:07.743 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:30:07.743 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:30:07.743 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:30:07.789 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:30:07.793 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:29:58.758+0000 7f1a6add4780 0 load: jerasure load: lrc 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:30:07.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T16:30:08.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:30:08.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:30:08.456 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:30:08.463 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:30:09.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:30:09.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:30:09.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:30:09.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:30:09.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:30:09.468 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:30:09.468 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:30:09.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:30:09.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:30:09.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:30:09.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:30:09.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:30:09.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:30:09.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:30:09.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:30:09.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:30:09.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:30:09.783 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:09.783 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:30:09.783 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:30:09.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:30:09.783 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:09.783 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:30:09.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-19T16:30:09.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-19T16:30:09.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-19T16:30:09.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:09.848 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:30:09.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-19T16:30:09.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-19T16:30:09.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-19T16:30:09.914 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:09.914 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:30:09.978 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444036 2026-02-19T16:30:09.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444036 2026-02-19T16:30:09.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-73014444036' 2026-02-19T16:30:09.979 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:09.979 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:30:10.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-19T16:30:10.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-19T16:30:10.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-73014444036 3-107374182403' 2026-02-19T16:30:10.043 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:10.043 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-19T16:30:10.043 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:10.044 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:30:10.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-19T16:30:10.045 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:10.045 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-19T16:30:10.046 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803780 2026-02-19T16:30:10.046 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-19T16:30:10.047 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:10.272 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-19T16:30:10.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:30:11.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:30:11.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:11.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-19T16:30:11.504 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:11.504 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-19T16:30:11.505 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:11.505 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:30:11.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-19T16:30:11.506 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:11.508 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-19T16:30:11.508 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-19T16:30:11.508 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607556 2026-02-19T16:30:11.508 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:30:11.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-19T16:30:11.736 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:11.736 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444036 2026-02-19T16:30:11.736 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:11.737 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:30:11.738 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444036 2026-02-19T16:30:11.738 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:11.739 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444036 2026-02-19T16:30:11.739 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 73014444036 2026-02-19T16:30:11.739 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-19T16:30:11.739 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:30:11.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444037 -lt 73014444036 2026-02-19T16:30:11.966 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:11.967 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182403 2026-02-19T16:30:11.967 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:11.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:30:11.968 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-19T16:30:11.969 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:11.969 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182403 2026-02-19T16:30:11.970 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-19T16:30:11.970 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 107374182403 2026-02-19T16:30:11.970 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:30:12.197 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-19T16:30:12.198 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:30:12.198 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:12.198 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:12.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:30:12.513 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:30:12.513 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:30:12.513 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:30:12.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:30:12.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:30:12.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:30:12.514 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:30:12.738 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:30:12.738 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:30:12.739 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:12.739 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:13.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:30:13.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:30:13.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:30:13.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-19T16:30:13.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-19T16:30:13.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-95545-0 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-95545-0 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:30:13.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-95545-0 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:30:13.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-95545-0 td/test-erasure-eio/ORIGINAL 2026-02-19T16:30:13.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-95545-0 td/test-erasure-eio 0 2026-02-19T16:30:13.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:30:13.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:30:13.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:30:13.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:30:13.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-95545-0 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-95545-0 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-0 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-0 2026-02-19T16:30:13.079 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:30:13.305 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:30:13.305 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:13.306 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:30:13.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:30:13.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T16:30:13.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:30:13.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-19T16:30:13.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:30:13.306 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T16:30:13.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:30:13.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T16:30:13.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:30:13.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T16:30:13.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:30:13.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:30:13.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:30:13.309 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:30:13.358 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:30:13.358 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:30:13.359 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:30:13.359 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:30:13.360 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-95545-0 0 2026-02-19T16:30:13.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-95545-0 2026-02-19T16:30:13.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:30:13.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:30:13.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-95545-0 2026-02-19T16:30:13.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:30:13.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:30:13.407 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-95545-0 td/test-erasure-eio/COPY 2026-02-19T16:30:13.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:30:13.425 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:30:13.426 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-19T16:30:13.426 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-95545-0 td/test-erasure-eio 1 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-95545-0 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-95545-0 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-0 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-0 2026-02-19T16:30:13.428 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:30:13.657 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T16:30:13.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:30:13.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T16:30:13.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:30:13.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T16:30:13.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:30:13.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:30:13.659 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:30:13.659 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T16:30:13.659 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:30:13.659 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:30:13.659 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:13.659 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:13.659 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:13.660 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:30:13.660 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:30:13.708 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:30:13.708 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:30:13.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:30:13.709 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-19T16:30:13.709 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:30:13.709 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:30:13.710 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:13.710 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:13.710 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:13.710 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:30:13.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:30:13.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-95545-0 1 2026-02-19T16:30:13.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T16:30:13.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-95545-0 fail 2026-02-19T16:30:13.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:30:13.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:30:13.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-95545-0 2026-02-19T16:30:13.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T16:30:13.758 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T16:30:13.759 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-95545-0 td/test-erasure-eio/COPY 2026-02-19T16:30:13.774 INFO:tasks.workunit.client.0.trial127.stderr:error getting pool-jerasure/obj-eio-95545-0: (5) Input/output error 2026-02-19T16:30:13.775 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T16:30:13.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:30:13.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:30:13.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:30:13.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:30:14.049 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:30:14.058 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:30:14.342 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:30:14.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:30:14.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:30:14.461 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:30:14.461 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:30:14.462 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:30:14.462 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:30:14.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:30:14.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:30:14.464 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:30:14.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:14.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:30:14.466 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:30:14.466 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:14.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:30:14.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:30:14.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:30:14.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:30:14.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:14.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:14.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:30:14.479 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:30:14.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:30:14.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:30:14.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:30:14.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:30:14.480 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:30:14.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:30:14.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:30:14.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:30:14.483 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:30:14.483 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:30:14.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:14.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:30:14.484 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:30:14.485 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:14.485 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:30:14.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:30:14.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:30:14.487 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:30:14.487 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:14.487 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:14.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:30:14.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:30:14.488 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:30:14.489 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:30:14.490 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:30:14.490 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:14.490 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:14.490 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:30:14.492 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:30:14.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:30:14.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:30:14.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:30:14.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:30:14.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:30:14.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:30:14.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:30:14.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:30:14.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:30:14.494 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:30:14.511 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:30:14.511 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:14.511 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:14.511 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:14.512 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:14.512 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:14.512 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:14.512 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:30:14.536 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:30:14.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:30:14.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:30:14.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:30:14.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:30:14.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:30:14.538 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:30:14.538 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:30:14.538 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:30:14.539 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:14.539 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:14.539 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:14.539 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:30:14.539 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:30:14.539 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:30:14.583 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:14.584 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:14.585 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:30:14.585 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:30:14.585 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:30:14.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:30:14.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:30:14.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:30:14.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:30:14.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:30:14.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:30:14.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:30:14.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:30:14.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:14.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:14.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:14.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:14.726 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:14.726 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:14.727 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:30:14.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:30:14.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:30:14.742 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:30:14.838 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:30:14.844 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:30:15.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:30:15.846 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:30:15.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:30:15.847 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:15.847 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:15.847 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:15.847 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:30:15.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:30:15.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:30:15.887 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:30:15.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:30:15.893 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:30:14.528+0000 7f358642fc00 0 load: jerasure load: lrc 2026-02-19T16:30:15.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-19T16:30:15.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-19T16:30:15.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-19T16:30:15.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T16:30:15.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:30:15.895 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T16:30:15.895 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:30:15.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:15.900 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:15.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:15.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:15.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:15.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:30:15.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:15.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=867c6f38-0d90-45cd-b5ca-713cf6cab9f3 2026-02-19T16:30:15.903 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 867c6f38-0d90-45cd-b5ca-713cf6cab9f3 2026-02-19T16:30:15.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 867c6f38-0d90-45cd-b5ca-713cf6cab9f3' 2026-02-19T16:30:15.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:15.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCXOpdpa8sWNhAA0btPGHXCTVmWzTBRj6bn3Q== 2026-02-19T16:30:15.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCXOpdpa8sWNhAA0btPGHXCTVmWzTBRj6bn3Q=="}' 2026-02-19T16:30:15.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 867c6f38-0d90-45cd-b5ca-713cf6cab9f3 -i td/test-erasure-eio/0/new.json 2026-02-19T16:30:16.011 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:16.018 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:30:16.019 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCXOpdpa8sWNhAA0btPGHXCTVmWzTBRj6bn3Q== --osd-uuid 867c6f38-0d90-45cd-b5ca-713cf6cab9f3 2026-02-19T16:30:16.035 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.034+0000 7f79a7251780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:16.037 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.035+0000 7f79a7251780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:16.037 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.036+0000 7f79a7251780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:16.038 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.036+0000 7f79a7251780 -1 bdev(0x268ac00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:16.038 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.036+0000 7f79a7251780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:30:16.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:30:16.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:16.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:30:16.503 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:30:16.503 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:16.608 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:30:16.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:30:16.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:16.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:16.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:16.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:16.624 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.622+0000 7fb61ba11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:16.625 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.623+0000 7fb61ba11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:16.626 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.624+0000 7fb61ba11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:16.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:30:16.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:16.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:30:16.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:16.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:16.706 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:16.707 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:16.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:16.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:16.708 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:30:16.805 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:16.828 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:16.827+0000 7fb61ba11780 -1 Falling back to public interface 2026-02-19T16:30:17.116 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:17.114+0000 7fb61ba11780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:30:17.807 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:17.807 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:17.807 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:17.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:17.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:17.808 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:30:18.034 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2181631282,v1:127.0.0.1:6803/2181631282] [v2:127.0.0.1:6804/2181631282,v1:127.0.0.1:6805/2181631282] exists,up 867c6f38-0d90-45cd-b5ca-713cf6cab9f3 2026-02-19T16:30:18.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:18.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:18.034 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:18.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:18.036 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:30:18.037 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:18.037 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e7d9ef05-183a-4efc-bfb0-47999ea700c3 2026-02-19T16:30:18.038 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e7d9ef05-183a-4efc-bfb0-47999ea700c3' 2026-02-19T16:30:18.038 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 e7d9ef05-183a-4efc-bfb0-47999ea700c3 2026-02-19T16:30:18.038 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:18.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCaOpdphS7FAhAAD1JZlplY+Z1nHtrCaiiDWQ== 2026-02-19T16:30:18.047 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCaOpdphS7FAhAAD1JZlplY+Z1nHtrCaiiDWQ=="}' 2026-02-19T16:30:18.048 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e7d9ef05-183a-4efc-bfb0-47999ea700c3 -i td/test-erasure-eio/1/new.json 2026-02-19T16:30:18.278 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:18.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:30:18.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCaOpdphS7FAhAAD1JZlplY+Z1nHtrCaiiDWQ== --osd-uuid e7d9ef05-183a-4efc-bfb0-47999ea700c3 2026-02-19T16:30:18.303 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.301+0000 7f57ecc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:18.305 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.302+0000 7f57ecc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:18.305 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.303+0000 7f57ecc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:18.306 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.303+0000 7f57ecc0e780 -1 bdev(0x37ccc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:18.306 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.303+0000 7f57ecc0e780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:30:18.624 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:30:18.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:18.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:30:18.626 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:30:18.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:18.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:30:18.943 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:30:18.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:18.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:18.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:18.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:18.958 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.955+0000 7f824722e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:18.959 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.957+0000 7f824722e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:18.960 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:18.958+0000 7f824722e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:19.099 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:19.096+0000 7f824722e780 -1 Falling back to public interface 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:19.166 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:19.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:19.167 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:19.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:19.394 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:19.392+0000 7f824722e780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:30:20.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:20.393 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:20.394 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:20.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:20.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:20.394 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:20.622 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3097368803,v1:127.0.0.1:6811/3097368803] [v2:127.0.0.1:6812/3097368803,v1:127.0.0.1:6813/3097368803] exists,up e7d9ef05-183a-4efc-bfb0-47999ea700c3 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:20.625 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:20.626 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:20.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=26d38128-d717-4e2b-af66-3fbada6b1fb7 2026-02-19T16:30:20.628 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 26d38128-d717-4e2b-af66-3fbada6b1fb7 2026-02-19T16:30:20.628 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 26d38128-d717-4e2b-af66-3fbada6b1fb7' 2026-02-19T16:30:20.628 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:20.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCcOpdpCrzVJRAAICJUdG42LSxsiOrkl5tl7g== 2026-02-19T16:30:20.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCcOpdpCrzVJRAAICJUdG42LSxsiOrkl5tl7g=="}' 2026-02-19T16:30:20.636 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 26d38128-d717-4e2b-af66-3fbada6b1fb7 -i td/test-erasure-eio/2/new.json 2026-02-19T16:30:20.866 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:30:20.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:30:20.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCcOpdpCrzVJRAAICJUdG42LSxsiOrkl5tl7g== --osd-uuid 26d38128-d717-4e2b-af66-3fbada6b1fb7 2026-02-19T16:30:20.890 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:20.887+0000 7fa33f00f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:20.891 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:20.889+0000 7fa33f00f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:20.892 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:20.890+0000 7fa33f00f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:20.892 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:20.890+0000 7fa33f00f780 -1 bdev(0x2d76c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:20.892 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:20.890+0000 7fa33f00f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:30:21.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:30:21.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:21.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:30:21.224 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:30:21.224 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:21.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:30:21.539 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:30:21.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:21.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:21.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:21.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:21.554 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:21.551+0000 7f0d27c6c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:21.555 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:21.553+0000 7f0d27c6c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:21.556 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:21.553+0000 7f0d27c6c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:21.726 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:21.723+0000 7f0d27c6c780 -1 Falling back to public interface 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:21.999 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:22.036 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:22.033+0000 7f0d27c6c780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:30:23.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:23.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:23.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:23.000 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:23.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:23.001 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:23.233 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/927792038,v1:127.0.0.1:6819/927792038] [v2:127.0.0.1:6820/927792038,v1:127.0.0.1:6821/927792038] exists,up 26d38128-d717-4e2b-af66-3fbada6b1fb7 2026-02-19T16:30:23.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:23.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:23.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:23.234 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:23.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:30:23.236 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:23.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=607b814f-d27c-47ab-9b00-7f62bc6bef1e 2026-02-19T16:30:23.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 607b814f-d27c-47ab-9b00-7f62bc6bef1e' 2026-02-19T16:30:23.236 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 607b814f-d27c-47ab-9b00-7f62bc6bef1e 2026-02-19T16:30:23.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:23.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCfOpdpUe2jDhAAiCIZ27wSYgTPU1Fe10aIrg== 2026-02-19T16:30:23.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCfOpdpUe2jDhAAiCIZ27wSYgTPU1Fe10aIrg=="}' 2026-02-19T16:30:23.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 607b814f-d27c-47ab-9b00-7f62bc6bef1e -i td/test-erasure-eio/3/new.json 2026-02-19T16:30:23.478 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:30:23.486 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:30:23.487 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCfOpdpUe2jDhAAiCIZ27wSYgTPU1Fe10aIrg== --osd-uuid 607b814f-d27c-47ab-9b00-7f62bc6bef1e 2026-02-19T16:30:23.504 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:23.501+0000 7f9fb1018780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:23.505 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:23.503+0000 7f9fb1018780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:23.506 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:23.504+0000 7f9fb1018780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:23.507 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:23.504+0000 7f9fb1018780 -1 bdev(0x36c2c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:23.507 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:23.504+0000 7f9fb1018780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:30:23.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:30:23.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:23.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:30:23.822 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:30:23.822 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:24.145 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:30:24.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:30:24.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:24.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:24.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:24.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:24.161 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:24.158+0000 7f5dc2c56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:24.162 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:24.160+0000 7f5dc2c56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:24.163 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:24.161+0000 7f5dc2c56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:24.324 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:24.322+0000 7f5dc2c56780 -1 Falling back to public interface 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:24.376 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:24.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:24.377 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:30:24.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:24.647 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:24.645+0000 7f5dc2c56780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:30:25.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:25.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:25.605 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:25.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:25.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:25.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:30:25.838 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/523833912,v1:127.0.0.1:6827/523833912] [v2:127.0.0.1:6828/523833912,v1:127.0.0.1:6829/523833912] exists,up 607b814f-d27c-47ab-9b00-7f62bc6bef1e 2026-02-19T16:30:25.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:25.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:25.840 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:25.840 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:30:25.840 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:30:25.840 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:30:25.841 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:25.841 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:25.841 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:25.841 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:30:25.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:30:25.841 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:30:25.884 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:30:25.888 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:30:25.890 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:30:16.829+0000 7fb61ba11780 0 load: jerasure load: lrc 2026-02-19T16:30:25.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-19T16:30:25.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T16:30:25.891 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:30:25.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:30:25.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T16:30:25.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:30:25.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T16:30:25.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:30:25.892 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T16:30:26.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:30:26.176 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:30:26.547 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:30:26.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:30:27.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:30:27.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:30:27.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:30:27.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:30:27.557 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:30:27.558 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:30:27.559 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:30:27.559 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:30:27.559 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:30:27.559 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:30:27.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:30:27.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:30:27.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:30:27.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:30:27.646 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:30:27.646 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:30:27.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:30:27.876 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:27.876 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:30:27.876 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:30:27.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:30:27.877 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:27.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:30:27.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-19T16:30:27.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-19T16:30:27.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-19T16:30:27.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:27.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:30:28.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-19T16:30:28.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-19T16:30:28.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852' 2026-02-19T16:30:28.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:28.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:30:28.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-19T16:30:28.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-19T16:30:28.072 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332' 2026-02-19T16:30:28.073 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:28.073 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:30:28.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116996 2026-02-19T16:30:28.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116996 2026-02-19T16:30:28.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332 3-115964116996' 2026-02-19T16:30:28.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:28.138 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-19T16:30:28.138 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:28.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:30:28.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-19T16:30:28.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:28.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-19T16:30:28.141 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-19T16:30:28.141 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803780 2026-02-19T16:30:28.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:28.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-19T16:30:28.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:30:29.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:30:29.371 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:29.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-19T16:30:29.598 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:29.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574852 2026-02-19T16:30:29.599 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:29.599 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:30:29.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-19T16:30:29.600 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:29.601 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574852 2026-02-19T16:30:29.602 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-19T16:30:29.602 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 55834574852 2026-02-19T16:30:29.602 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:30:29.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-19T16:30:29.827 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:29.827 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-19T16:30:29.827 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:29.828 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:30:29.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-19T16:30:29.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:29.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-19T16:30:29.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-19T16:30:29.830 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411332 2026-02-19T16:30:29.831 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:30:30.058 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-19T16:30:30.058 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:30.058 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116996 2026-02-19T16:30:30.059 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:30.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:30:30.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116996 2026-02-19T16:30:30.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:30.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116996 2026-02-19T16:30:30.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116996' 2026-02-19T16:30:30.061 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 115964116996 2026-02-19T16:30:30.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:30:30.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116996 -lt 115964116996 2026-02-19T16:30:30.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:30:30.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:30.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:30.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:30:30.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:30:30.605 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:30:30.605 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:30:30.606 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:30:30.606 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:30:30.606 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:30:30.606 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:30:30.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:30:30.833 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:30:30.833 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:30.833 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-95545-1 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-95545-1 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-95545-1 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:31.157 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:30:31.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:31.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:30:31.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-95545-1 td/test-erasure-eio/ORIGINAL 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-95545-1 td/test-erasure-eio 1 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-95545-1 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:30:31.175 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-95545-1 2026-02-19T16:30:31.176 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:30:31.176 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-1 2026-02-19T16:30:31.176 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-1 2026-02-19T16:30:31.176 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:30:31.405 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:30:31.405 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:31.406 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:30:31.406 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:30:31.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T16:30:31.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:30:31.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-19T16:30:31.406 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:30:31.406 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T16:30:31.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:30:31.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T16:30:31.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:30:31.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T16:30:31.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:30:31.408 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:30:31.408 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:30:31.409 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:30:31.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:30:31.457 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:30:31.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:30:31.458 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-19T16:30:31.458 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:30:31.458 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:30:31.458 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:31.458 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:31.458 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:31.459 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:30:31.459 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:30:31.459 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-95545-1 1 2026-02-19T16:30:31.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-95545-1 2026-02-19T16:30:31.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:30:31.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:30:31.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-95545-1 2026-02-19T16:30:31.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:30:31.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:30:31.507 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-95545-1 td/test-erasure-eio/COPY 2026-02-19T16:30:31.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:30:31.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:30:31.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-19T16:30:31.527 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-19T16:30:31.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-19T16:30:31.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-95545-1 td/test-erasure-eio 2 2026-02-19T16:30:31.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:30:31.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:30:31.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-95545-1 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=2 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-95545-1 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:30:31.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-1 2026-02-19T16:30:31.532 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-1 2026-02-19T16:30:31.532 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:30:31.761 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=2 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:30:31.762 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/2/type 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-19T16:30:31.764 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:31.765 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:31.765 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:31.765 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.2.asok 2026-02-19T16:30:31.765 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:30:31.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:30:31.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:30:31.813 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:30:31.814 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.2 2026-02-19T16:30:31.814 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-19T16:30:31.815 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-19T16:30:31.815 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:31.815 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:31.815 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:31.815 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.2.asok 2026-02-19T16:30:31.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:30:31.815 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-95545-1 2 2026-02-19T16:30:31.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T16:30:31.862 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-95545-1 2026-02-19T16:30:31.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:30:31.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:30:31.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-95545-1 2026-02-19T16:30:31.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:30:31.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:30:31.863 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-95545-1 td/test-erasure-eio/COPY 2026-02-19T16:30:31.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:30:31.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:30:31.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:30:31.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:30:31.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:30:31.882 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:30:32.155 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:30:32.164 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:30:32.443 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:30:32.452 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:30:32.453 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:30:32.562 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:30:32.563 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:30:32.563 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:30:32.564 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:30:32.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:30:32.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:30:32.566 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:30:32.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:32.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:30:32.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:30:32.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:32.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:30:32.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:30:32.569 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:30:32.578 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:30:32.578 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:32.578 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:32.578 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:30:32.579 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:30:32.580 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:30:32.580 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:30:32.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:30:32.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:30:32.580 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:30:32.582 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:30:32.582 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:30:32.583 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:30:32.583 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:30:32.585 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:30:32.585 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:30:32.585 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:30:32.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:32.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:30:32.586 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:30:32.586 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:30:32.587 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:30:32.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:30:32.589 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:30:32.590 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:30:32.590 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:32.590 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:32.590 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:30:32.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:30:32.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:30:32.591 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:30:32.592 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:30:32.592 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:32.592 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:32.592 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:30:32.593 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:30:32.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:30:32.594 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:30:32.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:30:32.613 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:30:32.613 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:32.613 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:32.613 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:32.614 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:32.614 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:32.614 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:32.614 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:30:32.639 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:30:32.640 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:30:32.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:30:32.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:30:32.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:30:32.641 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:30:32.641 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:30:32.641 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:30:32.641 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:30:32.642 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:32.643 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:32.643 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:32.643 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:30:32.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:30:32.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:30:32.684 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:30:32.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:30:32.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:30:32.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:30:32.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:30:32.686 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:30:32.686 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:30:32.686 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:30:32.687 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:32.687 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:32.687 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:32.687 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:30:32.687 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:30:32.687 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:30:32.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:30:32.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:30:32.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:30:32.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:30:32.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:30:32.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:30:32.729 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:30:32.828 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:30:32.828 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:32.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:32.829 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:32.829 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:32.829 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:32.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:32.830 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:30:32.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:30:32.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:30:32.845 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:30:32.944 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:30:32.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:30:33.953 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:30:33.953 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:30:33.953 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:30:33.953 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:33.953 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:33.954 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:33.954 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:30:33.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:30:33.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:30:33.994 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:30:33.998 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:30:33.999 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:30:32.630+0000 7f4476266c00 0 load: jerasure load: lrc 2026-02-19T16:30:34.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-19T16:30:34.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-19T16:30:34.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-19T16:30:34.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T16:30:34.000 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:30:34.000 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T16:30:34.001 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:34.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:34.004 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:30:34.005 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:34.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4f532d2d-48be-46d4-96d9-a7e2a2672a0c 2026-02-19T16:30:34.006 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4f532d2d-48be-46d4-96d9-a7e2a2672a0c' 2026-02-19T16:30:34.006 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 4f532d2d-48be-46d4-96d9-a7e2a2672a0c 2026-02-19T16:30:34.006 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:34.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCqOpdpfrnpABAAc8alREP6tCr1Lmm70glhwQ== 2026-02-19T16:30:34.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCqOpdpfrnpABAAc8alREP6tCr1Lmm70glhwQ=="}' 2026-02-19T16:30:34.017 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4f532d2d-48be-46d4-96d9-a7e2a2672a0c -i td/test-erasure-eio/0/new.json 2026-02-19T16:30:34.122 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:34.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:30:34.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCqOpdpfrnpABAAc8alREP6tCr1Lmm70glhwQ== --osd-uuid 4f532d2d-48be-46d4-96d9-a7e2a2672a0c 2026-02-19T16:30:34.148 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.146+0000 7f99031d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:34.149 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.148+0000 7f99031d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:34.150 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.148+0000 7f99031d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:34.150 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.149+0000 7f99031d8780 -1 bdev(0x3c18c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:34.150 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.149+0000 7f99031d8780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:30:34.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:30:34.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:34.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:30:34.557 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:30:34.557 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:34.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:30:34.659 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:30:34.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:34.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:34.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:34.659 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:34.675 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.673+0000 7f427ec31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:34.676 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.675+0000 7f427ec31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:34.677 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.676+0000 7f427ec31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:34.757 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:30:34.855 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:34.880 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:34.879+0000 7f427ec31780 -1 Falling back to public interface 2026-02-19T16:30:35.114 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:35.112+0000 7f427ec31780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:30:35.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:35.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:35.856 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:35.856 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:35.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:35.857 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:30:36.082 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/658887899,v1:127.0.0.1:6803/658887899] [v2:127.0.0.1:6804/658887899,v1:127.0.0.1:6805/658887899] exists,up 4f532d2d-48be-46d4-96d9-a7e2a2672a0c 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:36.083 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:36.084 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=670008b3-3829-4d10-b947-104b897d80c8 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 670008b3-3829-4d10-b947-104b897d80c8' 2026-02-19T16:30:36.085 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 670008b3-3829-4d10-b947-104b897d80c8 2026-02-19T16:30:36.086 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:36.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCsOpdpWTOcBRAAODIsrwtCaYDgrF1iy0pG5Q== 2026-02-19T16:30:36.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCsOpdpWTOcBRAAODIsrwtCaYDgrF1iy0pG5Q=="}' 2026-02-19T16:30:36.095 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 670008b3-3829-4d10-b947-104b897d80c8 -i td/test-erasure-eio/1/new.json 2026-02-19T16:30:36.323 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:36.330 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:30:36.331 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCsOpdpWTOcBRAAODIsrwtCaYDgrF1iy0pG5Q== --osd-uuid 670008b3-3829-4d10-b947-104b897d80c8 2026-02-19T16:30:36.347 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:36.346+0000 7f2c64408780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:36.348 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:36.347+0000 7f2c64408780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:36.349 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:36.348+0000 7f2c64408780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:36.350 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:36.348+0000 7f2c64408780 -1 bdev(0x3d72c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:36.350 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:36.348+0000 7f2c64408780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:30:36.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:30:36.690 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:36.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:30:36.691 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:30:36.691 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:37.008 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:30:37.009 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:30:37.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:37.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:37.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:37.010 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:37.023 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:37.021+0000 7f3e04c14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:37.024 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:37.022+0000 7f3e04c14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:37.025 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:37.023+0000 7f3e04c14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:37.162 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:37.160+0000 7f3e04c14780 -1 Falling back to public interface 2026-02-19T16:30:37.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:30:37.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:37.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:30:37.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:37.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:37.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:37.233 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:37.233 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:37.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:37.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:37.456 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:37.523 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:37.521+0000 7f3e04c14780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:30:38.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:38.458 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:38.458 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:38.459 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:38.459 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:38.459 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:38.688 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3276904542,v1:127.0.0.1:6811/3276904542] [v2:127.0.0.1:6812/3276904542,v1:127.0.0.1:6813/3276904542] exists,up 670008b3-3829-4d10-b947-104b897d80c8 2026-02-19T16:30:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:30:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:38.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:38.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:38.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=017e154c-61a8-40aa-b5df-19c623fce8d8 2026-02-19T16:30:38.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 017e154c-61a8-40aa-b5df-19c623fce8d8' 2026-02-19T16:30:38.696 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 017e154c-61a8-40aa-b5df-19c623fce8d8 2026-02-19T16:30:38.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:38.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCuOpdpOAbDKRAAhaFo2OEfo6dBrkCr70MeYw== 2026-02-19T16:30:38.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCuOpdpOAbDKRAAhaFo2OEfo6dBrkCr70MeYw=="}' 2026-02-19T16:30:38.702 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 017e154c-61a8-40aa-b5df-19c623fce8d8 -i td/test-erasure-eio/2/new.json 2026-02-19T16:30:38.933 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:30:38.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:30:38.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCuOpdpOAbDKRAAhaFo2OEfo6dBrkCr70MeYw== --osd-uuid 017e154c-61a8-40aa-b5df-19c623fce8d8 2026-02-19T16:30:38.957 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:38.956+0000 7fbdb5a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:38.958 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:38.957+0000 7fbdb5a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:38.959 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:38.958+0000 7fbdb5a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:38.960 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:38.958+0000 7fbdb5a65780 -1 bdev(0x3358c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:38.960 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:38.958+0000 7fbdb5a65780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:30:39.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:30:39.284 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:39.285 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:30:39.285 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:30:39.285 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:39.609 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:30:39.610 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:30:39.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:39.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:39.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:39.611 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:39.624 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:39.622+0000 7f76bbc2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:39.625 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:39.623+0000 7f76bbc2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:39.626 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:39.624+0000 7f76bbc2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:39.806 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:39.805+0000 7f76bbc2d780 -1 Falling back to public interface 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:39.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:40.064 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:40.155 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:40.153+0000 7f76bbc2d780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:30:41.065 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:41.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:41.066 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:41.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:41.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:41.066 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:41.295 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/67007744,v1:127.0.0.1:6819/67007744] [v2:127.0.0.1:6820/67007744,v1:127.0.0.1:6821/67007744] exists,up 017e154c-61a8-40aa-b5df-19c623fce8d8 2026-02-19T16:30:41.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:41.295 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:41.296 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:30:41.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:30:41.298 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:30:41.299 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=77963eff-a313-41f0-88f0-69b08b32e2d0 2026-02-19T16:30:41.299 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 77963eff-a313-41f0-88f0-69b08b32e2d0' 2026-02-19T16:30:41.299 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 77963eff-a313-41f0-88f0-69b08b32e2d0 2026-02-19T16:30:41.299 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:30:41.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCxOpdpDfxZEhAA8TPEcd/r3KzqLyd4uH90PQ== 2026-02-19T16:30:41.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCxOpdpDfxZEhAA8TPEcd/r3KzqLyd4uH90PQ=="}' 2026-02-19T16:30:41.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 77963eff-a313-41f0-88f0-69b08b32e2d0 -i td/test-erasure-eio/3/new.json 2026-02-19T16:30:41.538 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:30:41.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:30:41.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCxOpdpDfxZEhAA8TPEcd/r3KzqLyd4uH90PQ== --osd-uuid 77963eff-a313-41f0-88f0-69b08b32e2d0 2026-02-19T16:30:41.564 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:41.562+0000 7f41e2a42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:41.566 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:41.564+0000 7f41e2a42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:41.566 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:41.565+0000 7f41e2a42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:41.567 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:41.565+0000 7f41e2a42780 -1 bdev(0x2956c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:30:41.567 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:41.565+0000 7f41e2a42780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:30:41.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:30:41.898 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:30:41.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:30:41.899 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:30:41.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:30:42.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:30:42.218 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:30:42.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:42.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:30:42.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:30:42.219 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:30:42.232 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:42.230+0000 7fd8e6232780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:42.233 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:42.232+0000 7fd8e6232780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:42.234 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:42.232+0000 7fd8e6232780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:42.396 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:42.395+0000 7fd8e6232780 -1 Falling back to public interface 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:42.443 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:30:42.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:42.712 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:42.710+0000 7fd8e6232780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:30:43.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:43.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:43.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:43.672 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:30:43.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:43.673 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:30:43.901 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1041822437,v1:127.0.0.1:6827/1041822437] [v2:127.0.0.1:6828/1041822437,v1:127.0.0.1:6829/1041822437] exists,up 77963eff-a313-41f0-88f0-69b08b32e2d0 2026-02-19T16:30:43.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:43.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:30:43.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:30:43.948 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:30:43.952 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:30:43.954 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:30:34.881+0000 7f427ec31780 0 load: jerasure load: lrc 2026-02-19T16:30:43.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:30:43.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T16:30:44.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:30:44.247 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:30:44.647 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:30:44.657 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:30:45.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:30:45.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:30:45.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:30:45.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:30:45.658 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:30:45.659 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:30:45.659 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:30:45.659 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:30:45.659 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:30:45.659 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:30:45.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:30:45.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:30:45.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:30:45.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:30:45.746 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:30:45.746 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:30:45.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:30:45.975 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:45.975 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:30:45.975 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:30:45.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:30:45.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:45.975 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:30:46.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-19T16:30:46.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-19T16:30:46.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-19T16:30:46.040 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:46.041 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:30:46.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-19T16:30:46.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-19T16:30:46.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574853' 2026-02-19T16:30:46.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:46.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:30:46.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378627 2026-02-19T16:30:46.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378627 2026-02-19T16:30:46.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574853 2-81604378627' 2026-02-19T16:30:46.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:46.173 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:30:46.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149700 2026-02-19T16:30:46.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149700 2026-02-19T16:30:46.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574853 2-81604378627 3-111669149700' 2026-02-19T16:30:46.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:46.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-19T16:30:46.237 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:46.238 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:30:46.238 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-19T16:30:46.239 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:46.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-19T16:30:46.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-19T16:30:46.240 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803780 2026-02-19T16:30:46.240 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:46.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-19T16:30:46.467 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:30:47.468 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:30:47.468 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:47.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-19T16:30:47.696 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:47.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-19T16:30:47.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:47.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:30:47.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-19T16:30:47.698 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:47.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-19T16:30:47.699 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-19T16:30:47.699 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 55834574853 2026-02-19T16:30:47.700 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:30:47.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-19T16:30:47.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:47.924 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378627 2026-02-19T16:30:47.924 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:47.925 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:30:47.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378627 2026-02-19T16:30:47.926 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:47.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378627 2026-02-19T16:30:47.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378627' 2026-02-19T16:30:47.928 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 81604378627 2026-02-19T16:30:47.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:30:48.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378628 -lt 81604378627 2026-02-19T16:30:48.154 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:48.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149700 2026-02-19T16:30:48.155 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:48.156 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:30:48.156 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149700 2026-02-19T16:30:48.156 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:48.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149700 2026-02-19T16:30:48.158 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-19T16:30:48.158 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149700 2026-02-19T16:30:48.158 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:30:48.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-19T16:30:48.389 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:30:48.389 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:48.389 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:48.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:30:48.701 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:30:48.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:30:48.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:30:48.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:30:48.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:30:48.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:30:48.702 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:30:48.924 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:30:48.924 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:30:48.925 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:48.925 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:49.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-95545-1 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-95545-1 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-95545-1 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:30:49.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-95545-1 td/test-erasure-eio/ORIGINAL 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-95545-1 td/test-erasure-eio 1 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-95545-1 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-19T16:30:49.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-19T16:30:49.259 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-19T16:30:49.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-95545-1 2026-02-19T16:30:49.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:30:49.259 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-95545-1 2026-02-19T16:30:49.259 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-95545-1 2026-02-19T16:30:49.259 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:30:49.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:30:49.491 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:49.491 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:30:49.491 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-95545-1 remove 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-95545-1 remove 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T16:30:49.492 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:30:49.493 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:30:49.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:30:49.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:30:49.493 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:30:49.800 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:30:49.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-95545-1 remove 2026-02-19T16:30:49.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:30:49.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:30:49.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T16:30:49.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:30:49.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T16:30:49.801 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-95545-1 remove 2026-02-19T16:30:49.898 INFO:tasks.workunit.client.0.trial127.stdout:remove 1#2:b6eee49d:::obj-remove-95545-1:head# 2026-02-19T16:30:49.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T16:30:49.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:49.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:49.942 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:49.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:30:49.944 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T16:30:49.945 INFO:tasks.workunit.client.0.trial127.stderr:start osd.1 2026-02-19T16:30:49.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:49.945 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T16:30:49.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T16:30:49.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:30:49.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:30:49.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:30:49.954 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:49.953+0000 7f8a1c885780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:49.956 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:49.954+0000 7f8a1c885780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:49.957 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:49.955+0000 7f8a1c885780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:50.139 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:50.138+0000 7f8a1c885780 -1 Falling back to public interface 2026-02-19T16:30:50.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T16:30:50.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:50.172 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:30:50.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:50.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:50.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:50.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:50.173 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:30:50.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:50.173 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:50.400 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:50.516 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:50.515+0000 7f8a1c885780 -1 osd.1 33 log_to_monitors true 2026-02-19T16:30:51.401 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:51.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:51.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:51.402 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:51.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:51.402 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:30:51.626 INFO:tasks.workunit.client.0.trial127.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [13,33) [v2:127.0.0.1:6810/3802786849,v1:127.0.0.1:6811/3802786849] [v2:127.0.0.1:6812/3802786849,v1:127.0.0.1:6813/3802786849] exists,up 670008b3-3829-4d10-b947-104b897d80c8 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:30:51.627 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:30:51.628 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:30:51.628 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:30:51.628 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:30:51.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:30:51.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:30:51.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:30:51.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:30:51.714 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:30:51.714 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:30:51.940 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:30:51.940 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:51.940 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:30:51.941 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:30:51.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:30:51.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:51.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:30:52.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T16:30:52.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T16:30:52.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T16:30:52.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:52.007 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:30:52.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789955 2026-02-19T16:30:52.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789955 2026-02-19T16:30:52.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-158913789955' 2026-02-19T16:30:52.075 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:52.075 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:30:52.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378631 2026-02-19T16:30:52.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378631 2026-02-19T16:30:52.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-158913789955 2-81604378631' 2026-02-19T16:30:52.137 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:52.138 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:30:52.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149703 2026-02-19T16:30:52.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149703 2026-02-19T16:30:52.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-158913789955 2-81604378631 3-111669149703' 2026-02-19T16:30:52.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:52.204 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-19T16:30:52.204 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:52.205 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:30:52.206 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T16:30:52.206 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:52.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-19T16:30:52.207 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T16:30:52.207 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803784 2026-02-19T16:30:52.208 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:52.434 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-19T16:30:52.434 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:30:53.435 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:30:53.436 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:53.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T16:30:53.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:53.663 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-158913789955 2026-02-19T16:30:53.664 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:53.664 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:30:53.665 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-158913789955 2026-02-19T16:30:53.665 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:53.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=158913789955 2026-02-19T16:30:53.666 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 158913789955' 2026-02-19T16:30:53.666 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 158913789955 2026-02-19T16:30:53.667 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:30:53.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-19T16:30:53.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:53.894 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378631 2026-02-19T16:30:53.895 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:53.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:30:53.896 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378631 2026-02-19T16:30:53.896 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:53.897 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378631 2026-02-19T16:30:53.897 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378631' 2026-02-19T16:30:53.897 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 81604378631 2026-02-19T16:30:53.897 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:30:54.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378631 -lt 81604378631 2026-02-19T16:30:54.126 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:54.127 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149703 2026-02-19T16:30:54.127 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:54.128 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:30:54.128 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149703 2026-02-19T16:30:54.128 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:54.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149703 2026-02-19T16:30:54.129 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149703' 2026-02-19T16:30:54.130 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149703 2026-02-19T16:30:54.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:30:54.362 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149703 2026-02-19T16:30:54.362 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:30:54.362 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:54.362 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:54.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:30:54.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:30:54.680 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:30:54.680 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:30:54.681 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:30:54.681 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:30:54.681 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:30:54.681 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:30:54.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:30:54.909 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:30:54.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:30:54.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:30:55.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:30:55.222 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-95545-1 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-95545-1 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:30:55.223 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-95545-1 td/test-erasure-eio/COPY 2026-02-19T16:30:55.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:30:55.240 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:30:55.241 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-19T16:30:55.242 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-19T16:30:55.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-19T16:30:55.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-95545-1 td/test-erasure-eio 2 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-95545-1 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-19T16:30:55.245 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-19T16:30:55.246 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-19T16:30:55.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-95545-1 2026-02-19T16:30:55.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:30:55.246 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-95545-1 2026-02-19T16:30:55.246 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-95545-1 2026-02-19T16:30:55.246 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:30:55.475 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:30:55.475 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:55.475 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-95545-1 remove 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-95545-1 remove 2026-02-19T16:30:55.476 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:30:55.477 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:30:55.781 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:30:55.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-95545-1 remove 2026-02-19T16:30:55.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T16:30:55.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T16:30:55.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T16:30:55.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T16:30:55.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T16:30:55.782 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-95545-1 remove 2026-02-19T16:30:55.889 INFO:tasks.workunit.client.0.trial127.stdout:remove 2#2:b6eee49d:::obj-remove-95545-1:head# 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:30:55.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:30:55.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:30:55.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:30:55.932 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:30:55.932 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:30:55.932 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:30:55.932 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:start osd.2 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:30:55.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:30:55.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:30:55.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:30:55.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:30:55.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:30:55.947 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:55.945+0000 7f4b8f016780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:55.949 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:55.946+0000 7f4b8f016780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:55.950 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:55.947+0000 7f4b8f016780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:30:56.084 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:56.081+0000 7f4b8f016780 -1 Falling back to public interface 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:0 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:56.160 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:56.387 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:30:56.422 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:30:56.420+0000 7f4b8f016780 -1 osd.2 39 log_to_monitors true 2026-02-19T16:30:57.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:30:57.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:30:57.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:30:57.388 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:57.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:30:57.388 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:30:57.619 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 up in weight 1 up_from 42 up_thru 34 down_at 39 last_clean_interval [19,39) [v2:127.0.0.1:6818/3587747006,v1:127.0.0.1:6819/3587747006] [v2:127.0.0.1:6820/3587747006,v1:127.0.0.1:6821/3587747006] exists,up 017e154c-61a8-40aa-b5df-19c623fce8d8 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:30:57.620 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:30:57.621 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:30:57.621 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:30:57.621 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:30:57.621 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:30:57.621 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:30:57.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:30:57.709 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:30:57.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:30:57.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:30:57.710 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:30:57.710 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:30:57.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:30:57.936 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:30:57.936 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:30:57.936 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:30:57.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:30:57.936 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:57.936 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:30:58.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-19T16:30:58.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-19T16:30:58.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-19T16:30:58.003 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:58.003 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:30:58.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789958 2026-02-19T16:30:58.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789958 2026-02-19T16:30:58.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-158913789958' 2026-02-19T16:30:58.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:58.070 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:30:58.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626435 2026-02-19T16:30:58.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626435 2026-02-19T16:30:58.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-158913789958 2-180388626435' 2026-02-19T16:30:58.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:30:58.137 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:30:58.200 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149706 2026-02-19T16:30:58.201 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149706 2026-02-19T16:30:58.201 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-158913789958 2-180388626435 3-111669149706' 2026-02-19T16:30:58.201 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:58.201 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-19T16:30:58.201 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:58.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:30:58.202 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-19T16:30:58.203 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:58.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-19T16:30:58.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-19T16:30:58.204 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.0 seq 25769803787 2026-02-19T16:30:58.204 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:58.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-19T16:30:58.432 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:30:59.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:30:59.434 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:30:59.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-19T16:30:59.662 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:59.662 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-158913789958 2026-02-19T16:30:59.662 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:59.663 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:30:59.664 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-158913789958 2026-02-19T16:30:59.664 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:59.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=158913789958 2026-02-19T16:30:59.665 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 158913789958' 2026-02-19T16:30:59.665 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.1 seq 158913789958 2026-02-19T16:30:59.665 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:30:59.897 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 158913789958 -lt 158913789958 2026-02-19T16:30:59.897 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:30:59.898 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-180388626435 2026-02-19T16:30:59.898 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:30:59.899 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:30:59.899 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-180388626435 2026-02-19T16:30:59.900 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:30:59.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626435 2026-02-19T16:30:59.901 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 180388626435' 2026-02-19T16:30:59.901 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.2 seq 180388626435 2026-02-19T16:30:59.901 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:31:00.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-19T16:31:00.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:00.124 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149706 2026-02-19T16:31:00.124 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:00.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:31:00.126 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149706 2026-02-19T16:31:00.126 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:00.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149706 2026-02-19T16:31:00.127 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149706' 2026-02-19T16:31:00.127 INFO:tasks.workunit.client.0.trial127.stderr:waiting osd.3 seq 111669149706 2026-02-19T16:31:00.127 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:31:00.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149707 -lt 111669149706 2026-02-19T16:31:00.356 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:31:00.356 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:00.356 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:00.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:31:00.677 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:31:00.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:31:00.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:31:00.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:31:00.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:31:00.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:31:00.677 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:31:00.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:31:00.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:31:00.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:00.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-95545-1 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-95545-1 2026-02-19T16:31:01.217 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:31:01.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:31:01.218 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-95545-1 td/test-erasure-eio/COPY 2026-02-19T16:31:01.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:31:01.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:31:01.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-19T16:31:01.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:31:01.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:31:01.237 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:31:01.508 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:31:01.516 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:31:01.794 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:31:01.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:31:01.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:31:01.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:31:01.803 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:31:01.804 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:01.804 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:01.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:01.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:01.804 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:01.912 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:01.913 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:31:01.913 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:31:01.914 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:31:01.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:31:01.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:31:01.915 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:31:01.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:01.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:31:01.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:31:01.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:01.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:01.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:31:01.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:31:01.929 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:31:01.929 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:01.929 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:01.929 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:31:01.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:01.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:01.933 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:01.933 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:31:01.934 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:31:01.934 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:31:01.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:31:01.935 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:31:01.935 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:31:01.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:01.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:31:01.937 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:31:01.937 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:01.938 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:01.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:31:01.939 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:31:01.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:31:01.940 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:01.941 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:01.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:31:01.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:31:01.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:31:01.941 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:31:01.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:31:01.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:01.943 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:01.943 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:31:01.944 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:31:01.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:31:01.963 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:31:01.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:01.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:01.964 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:01.964 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:01.964 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:01.964 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:01.964 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:31:01.988 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:31:01.989 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:31:01.989 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:31:01.989 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:31:01.989 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:31:01.990 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:31:01.990 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:31:01.990 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:31:01.990 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:31:01.990 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:01.990 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:01.990 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:01.991 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:31:01.991 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:31:01.991 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:31:02.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:31:02.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:31:02.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:31:02.035 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:31:02.036 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:31:02.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:31:02.079 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:31:02.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:31:02.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:31:02.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:31:02.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:31:02.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:31:02.179 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:31:02.179 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:02.179 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:02.179 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:02.180 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:02.180 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:02.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:02.180 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:31:02.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:31:02.196 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:31:02.196 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:31:02.291 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:31:02.297 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:31:03.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:31:03.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:31:03.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:31:03.300 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:03.300 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:03.300 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:03.300 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:31:03.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:31:03.301 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:31:03.341 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:31:03.345 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:31:03.347 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:31:01.979+0000 7fd0392fbc00 0 load: jerasure load: lrc 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T16:31:03.348 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T16:31:03.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:03.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:31:03.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:03.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:03.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:31:03.351 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:03.352 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=448b5d5d-74f9-4ead-bb01-31c1a25746cb 2026-02-19T16:31:03.353 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 448b5d5d-74f9-4ead-bb01-31c1a25746cb' 2026-02-19T16:31:03.354 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 448b5d5d-74f9-4ead-bb01-31c1a25746cb 2026-02-19T16:31:03.354 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:03.364 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDHOpdpNtSZFRAAO7o0vjWWWt+V0G5t88loGg== 2026-02-19T16:31:03.364 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDHOpdpNtSZFRAAO7o0vjWWWt+V0G5t88loGg=="}' 2026-02-19T16:31:03.364 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 448b5d5d-74f9-4ead-bb01-31c1a25746cb -i td/test-erasure-eio/0/new.json 2026-02-19T16:31:03.466 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:03.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:31:03.474 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDHOpdpNtSZFRAAO7o0vjWWWt+V0G5t88loGg== --osd-uuid 448b5d5d-74f9-4ead-bb01-31c1a25746cb 2026-02-19T16:31:03.490 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:03.489+0000 7f8769c2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:03.492 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:03.491+0000 7f8769c2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:03.493 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:03.491+0000 7f8769c2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:03.493 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:03.492+0000 7f8769c2e780 -1 bdev(0x38f0c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:03.493 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:03.492+0000 7f8769c2e780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:31:03.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:31:03.930 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:03.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:31:03.931 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:31:03.931 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:04.032 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:31:04.033 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:31:04.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:04.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:04.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:04.033 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:04.048 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:04.046+0000 7f61e4c42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:04.049 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:04.048+0000 7f61e4c42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:04.050 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:04.049+0000 7f61e4c42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:04.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:31:04.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:04.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:31:04.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:04.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:04.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:04.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:04.133 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:04.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:04.133 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:31:04.230 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:04.250 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:04.249+0000 7f61e4c42780 -1 Falling back to public interface 2026-02-19T16:31:04.509 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:04.507+0000 7f61e4c42780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:31:05.231 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:05.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:05.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:05.232 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:05.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:05.232 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:31:05.467 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3459840300,v1:127.0.0.1:6803/3459840300] [v2:127.0.0.1:6804/3459840300,v1:127.0.0.1:6805/3459840300] exists,up 448b5d5d-74f9-4ead-bb01-31c1a25746cb 2026-02-19T16:31:05.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:05.470 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:05.471 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:05.472 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4514c2d2-7c67-4389-b85a-9bff5e02eac9 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4514c2d2-7c67-4389-b85a-9bff5e02eac9' 2026-02-19T16:31:05.473 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 4514c2d2-7c67-4389-b85a-9bff5e02eac9 2026-02-19T16:31:05.474 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:05.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDJOpdpUmuaHBAA2whxK4rcmiNLFDnMAnSMow== 2026-02-19T16:31:05.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDJOpdpUmuaHBAA2whxK4rcmiNLFDnMAnSMow=="}' 2026-02-19T16:31:05.481 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4514c2d2-7c67-4389-b85a-9bff5e02eac9 -i td/test-erasure-eio/1/new.json 2026-02-19T16:31:05.716 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:05.725 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:31:05.726 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJOpdpUmuaHBAA2whxK4rcmiNLFDnMAnSMow== --osd-uuid 4514c2d2-7c67-4389-b85a-9bff5e02eac9 2026-02-19T16:31:05.742 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:05.740+0000 7fcdad633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:05.743 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:05.742+0000 7fcdad633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:05.744 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:05.743+0000 7fcdad633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:05.744 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:05.743+0000 7fcdad633780 -1 bdev(0x39f0c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:05.744 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:05.743+0000 7fcdad633780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:31:06.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:31:06.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:06.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:31:06.062 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:31:06.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:06.379 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:31:06.379 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:31:06.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:06.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:06.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:06.380 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:06.395 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:06.393+0000 7f3087df9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:06.396 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:06.394+0000 7f3087df9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:06.397 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:06.395+0000 7f3087df9780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:06.549 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:06.548+0000 7f3087df9780 -1 Falling back to public interface 2026-02-19T16:31:06.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:31:06.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:06.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:31:06.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:06.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:06.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:06.606 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:06.607 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:06.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:06.607 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:31:06.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:06.929 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:06.927+0000 7f3087df9780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:31:07.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:07.832 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:07.832 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:07.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:07.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:07.833 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:31:08.056 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4256550482,v1:127.0.0.1:6811/4256550482] [v2:127.0.0.1:6812/4256550482,v1:127.0.0.1:6813/4256550482] exists,up 4514c2d2-7c67-4389-b85a-9bff5e02eac9 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:08.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:08.060 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:08.061 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:31:08.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:08.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d4883d79-11ec-4fbc-842e-43c6c3d9e84f 2026-02-19T16:31:08.062 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d4883d79-11ec-4fbc-842e-43c6c3d9e84f' 2026-02-19T16:31:08.062 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 d4883d79-11ec-4fbc-842e-43c6c3d9e84f 2026-02-19T16:31:08.062 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:08.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDMOpdpLyAeBBAAosz2P0pllDVUNeiSV3sVpg== 2026-02-19T16:31:08.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDMOpdpLyAeBBAAosz2P0pllDVUNeiSV3sVpg=="}' 2026-02-19T16:31:08.071 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d4883d79-11ec-4fbc-842e-43c6c3d9e84f -i td/test-erasure-eio/2/new.json 2026-02-19T16:31:08.298 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:31:08.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:31:08.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDMOpdpLyAeBBAAosz2P0pllDVUNeiSV3sVpg== --osd-uuid d4883d79-11ec-4fbc-842e-43c6c3d9e84f 2026-02-19T16:31:08.323 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.321+0000 7f092aa2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:08.324 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.323+0000 7f092aa2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:08.325 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.323+0000 7f092aa2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:08.325 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.324+0000 7f092aa2f780 -1 bdev(0x383ac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:08.325 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.324+0000 7f092aa2f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:31:08.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:31:08.650 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:08.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:31:08.651 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:31:08.651 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:08.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:31:08.974 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:31:08.974 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:08.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:08.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:08.975 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:08.988 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.987+0000 7fadd2618780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:08.989 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.988+0000 7fadd2618780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:08.990 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:08.989+0000 7fadd2618780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:09.125 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:09.123+0000 7fadd2618780 -1 Falling back to public interface 2026-02-19T16:31:09.202 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:31:09.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:09.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:31:09.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:09.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:09.203 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:09.203 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:09.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:09.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:09.204 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:09.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:09.500 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:09.499+0000 7fadd2618780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:31:10.435 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:10.435 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:10.435 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:10.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:10.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:10.436 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:10.667 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2138450774,v1:127.0.0.1:6819/2138450774] [v2:127.0.0.1:6820/2138450774,v1:127.0.0.1:6821/2138450774] exists,up d4883d79-11ec-4fbc-842e-43c6c3d9e84f 2026-02-19T16:31:10.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:10.669 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:10.670 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:10.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:31:10.672 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:10.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ebf9b040-ad32-4702-8c55-387a32fb900e 2026-02-19T16:31:10.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 ebf9b040-ad32-4702-8c55-387a32fb900e' 2026-02-19T16:31:10.672 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 ebf9b040-ad32-4702-8c55-387a32fb900e 2026-02-19T16:31:10.672 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:10.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDOOpdpEYiEKBAAvZ8BUYtH2ndOk4K+4XN1EQ== 2026-02-19T16:31:10.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDOOpdpEYiEKBAAvZ8BUYtH2ndOk4K+4XN1EQ=="}' 2026-02-19T16:31:10.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ebf9b040-ad32-4702-8c55-387a32fb900e -i td/test-erasure-eio/3/new.json 2026-02-19T16:31:10.913 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:31:10.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:31:10.922 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDOOpdpEYiEKBAAvZ8BUYtH2ndOk4K+4XN1EQ== --osd-uuid ebf9b040-ad32-4702-8c55-387a32fb900e 2026-02-19T16:31:10.938 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:10.937+0000 7f2f547d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:10.940 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:10.938+0000 7f2f547d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:10.940 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:10.939+0000 7f2f547d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:10.941 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:10.939+0000 7f2f547d8780 -1 bdev(0x408ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:10.941 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:10.939+0000 7f2f547d8780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:31:11.268 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:31:11.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:11.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:31:11.269 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:31:11.269 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:11.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:31:11.596 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:31:11.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:11.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:11.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:11.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:11.612 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:11.610+0000 7f5cabc11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:11.613 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:11.611+0000 7f5cabc11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:11.614 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:11.612+0000 7f5cabc11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:11.759 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:11.758+0000 7f5cabc11780 -1 Falling back to public interface 2026-02-19T16:31:11.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:31:11.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:11.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:31:11.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:11.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:11.830 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:11.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:11.831 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:11.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:11.831 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:31:12.057 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:12.145 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:12.143+0000 7f5cabc11780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:31:13.058 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:13.058 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:13.058 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:13.059 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:13.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:13.059 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:31:13.286 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2042491052,v1:127.0.0.1:6827/2042491052] [v2:127.0.0.1:6828/2042491052,v1:127.0.0.1:6829/2042491052] exists,up ebf9b040-ad32-4702-8c55-387a32fb900e 2026-02-19T16:31:13.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:13.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:13.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:13.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:31:13.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:31:13.287 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:31:13.288 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:13.288 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:13.288 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:13.288 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:31:13.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:31:13.288 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:31:13.332 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:31:13.336 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:31:13.338 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:31:04.251+0000 7f61e4c42780 0 load: jerasure load: lrc 2026-02-19T16:31:13.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-19T16:31:13.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T16:31:13.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:31:13.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:31:13.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T16:31:13.339 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:31:13.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T16:31:13.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:31:13.340 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T16:31:13.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:31:13.630 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:31:14.019 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:31:14.027 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:31:15.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:31:15.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:31:15.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:31:15.029 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:31:15.029 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:31:15.029 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:31:15.029 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:31:15.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:31:15.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:31:15.030 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:31:15.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:31:15.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:31:15.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:31:15.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:31:15.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:31:15.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:31:15.349 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:31:15.349 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:15.350 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:31:15.350 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:31:15.350 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:31:15.350 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:15.350 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:31:15.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T16:31:15.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T16:31:15.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T16:31:15.415 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:15.416 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:31:15.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-19T16:31:15.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-19T16:31:15.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-19T16:31:15.482 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:15.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:31:15.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T16:31:15.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T16:31:15.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411331' 2026-02-19T16:31:15.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:15.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:31:15.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T16:31:15.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T16:31:15.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411331 3-111669149699' 2026-02-19T16:31:15.613 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:15.614 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-19T16:31:15.614 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:15.615 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:31:15.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T16:31:15.616 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:15.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-19T16:31:15.617 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T16:31:15.618 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803781 2026-02-19T16:31:15.618 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:15.847 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-19T16:31:15.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:31:16.848 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:31:16.849 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:17.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-19T16:31:17.077 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:17.077 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-19T16:31:17.077 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:17.078 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:31:17.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-19T16:31:17.079 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:17.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-19T16:31:17.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-19T16:31:17.081 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607556 2026-02-19T16:31:17.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:31:17.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-19T16:31:17.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:17.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-19T16:31:17.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:17.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:31:17.309 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T16:31:17.309 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:17.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-19T16:31:17.310 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411331 2026-02-19T16:31:17.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T16:31:17.311 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:31:17.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T16:31:17.537 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:17.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-19T16:31:17.537 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:17.538 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:31:17.539 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T16:31:17.539 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:17.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-19T16:31:17.540 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T16:31:17.540 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149699 2026-02-19T16:31:17.540 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:31:17.764 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-19T16:31:17.764 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:31:17.764 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:17.765 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:18.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:31:18.086 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:31:18.086 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:31:18.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:31:18.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:31:18.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:31:18.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:31:18.087 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:31:18.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:31:18.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:31:18.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:18.310 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-95545-0 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-95545-0 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-95545-0 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:31:18.629 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-95545-0 td/test-erasure-eio/ORIGINAL 2026-02-19T16:31:18.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-95545-0 td/test-erasure-eio 0 2026-02-19T16:31:18.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:31:18.647 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-95545-0 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-95545-0 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:31:18.648 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-0 2026-02-19T16:31:18.649 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-0 2026-02-19T16:31:18.649 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:31:18.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:31:18.877 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:18.877 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:31:18.877 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:31:18.877 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T16:31:18.877 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:31:18.877 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-19T16:31:18.878 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:31:18.878 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T16:31:18.879 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:18.880 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:18.881 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:31:18.881 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:31:18.926 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:31:18.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:31:18.927 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:31:18.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-19T16:31:18.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T16:31:18.927 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T16:31:18.928 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:18.928 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:18.928 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:18.928 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.3.asok 2026-02-19T16:31:18.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:31:18.928 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-95545-0 0 2026-02-19T16:31:18.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-95545-0 2026-02-19T16:31:18.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:31:18.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:31:18.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-95545-0 2026-02-19T16:31:18.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:31:18.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:31:18.976 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-95545-0 td/test-erasure-eio/COPY 2026-02-19T16:31:18.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:31:18.994 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:31:18.995 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-19T16:31:18.996 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-95545-0 2026-02-19T16:31:18.996 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:31:18.996 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-0 2026-02-19T16:31:18.996 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-0 2026-02-19T16:31:18.996 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:31:19.226 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:31:19.226 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:19.227 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:19.534 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:19.534 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-19T16:31:19.812 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already out. 2026-02-19T16:31:19.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-95545-0 2026-02-19T16:31:19.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:31:19.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-0 2026-02-19T16:31:19.821 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-19T16:31:19.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-0 2026-02-19T16:31:19.822 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:31:20.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:31:20.049 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:20.049 INFO:tasks.workunit.client.0.trial127.stderr:0' 2026-02-19T16:31:20.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-19T16:31:20.049 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-19T16:31:20.299 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already in. 2026-02-19T16:31:20.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-19T16:31:20.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:20.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:31:20.310 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:31:20.311 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:20.312 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:20.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:20.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:20.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:31:20.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:31:20.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:31:20.313 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:31:20.313 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:31:20.314 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:20.314 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:31:20.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:31:20.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:31:20.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:31:20.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:31:20.324 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:20.323+0000 7fcc41631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:20.326 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:20.324+0000 7fcc41631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:20.326 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:20.325+0000 7fcc41631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:20.477 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:20.476+0000 7fcc41631780 -1 Falling back to public interface 2026-02-19T16:31:20.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:31:20.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:20.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:20.766 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:20.851 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:20.849+0000 7fcc41631780 -1 osd.2 33 log_to_monitors true 2026-02-19T16:31:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:21.767 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:21.767 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:21.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:21.768 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:21.992 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 42 up_thru 19 down_at 34 last_clean_interval [18,33) [v2:127.0.0.1:6818/3714026091,v1:127.0.0.1:6819/3714026091] [v2:127.0.0.1:6820/3714026091,v1:127.0.0.1:6821/3714026091] exists,up d4883d79-11ec-4fbc-842e-43c6c3d9e84f 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:31:21.993 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:31:21.994 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:31:21.994 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:31:21.994 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:31:21.994 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:31:21.994 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:31:22.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:31:22.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:31:22.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:31:22.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:31:22.082 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:31:22.082 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:31:22.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:31:22.304 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:22.304 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:31:22.304 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:31:22.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:31:22.304 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:22.304 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:31:22.369 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T16:31:22.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T16:31:22.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T16:31:22.370 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:22.370 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:31:22.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-19T16:31:22.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-19T16:31:22.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607559' 2026-02-19T16:31:22.433 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:22.434 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:31:22.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626435 2026-02-19T16:31:22.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626435 2026-02-19T16:31:22.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607559 2-180388626435' 2026-02-19T16:31:22.501 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:22.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:31:22.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T16:31:22.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T16:31:22.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607559 2-180388626435 3-111669149702' 2026-02-19T16:31:22.565 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:22.566 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-19T16:31:22.566 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:22.567 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:31:22.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T16:31:22.567 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:22.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-19T16:31:22.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T16:31:22.569 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803785 2026-02-19T16:31:22.569 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:22.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-19T16:31:22.795 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:31:23.796 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:31:23.797 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:24.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-19T16:31:24.024 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:31:25.025 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:31:25.025 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:25.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T16:31:25.254 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:25.255 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607559 2026-02-19T16:31:25.255 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:25.256 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:31:25.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-19T16:31:25.256 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:25.257 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607559 2026-02-19T16:31:25.258 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-19T16:31:25.258 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607559 2026-02-19T16:31:25.258 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:31:25.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607560 -lt 51539607559 2026-02-19T16:31:25.495 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:25.495 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-180388626435 2026-02-19T16:31:25.496 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:25.496 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:31:25.497 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-180388626435 2026-02-19T16:31:25.497 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:25.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626435 2026-02-19T16:31:25.498 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 180388626435' 2026-02-19T16:31:25.498 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 180388626435 2026-02-19T16:31:25.499 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:31:25.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-19T16:31:25.727 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:25.728 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-19T16:31:25.728 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:25.728 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:31:25.729 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T16:31:25.729 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:25.730 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-19T16:31:25.731 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T16:31:25.731 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149702 2026-02-19T16:31:25.731 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:31:25.960 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149702 2026-02-19T16:31:25.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:31:25.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:25.961 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:26.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:31:26.273 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:31:26.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:31:26.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:31:26.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:31:26.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:31:26.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:31:26.274 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:31:26.502 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:31:26.502 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:31:26.503 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:26.503 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:26.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:31:26.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:31:26.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:31:26.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:31:26.824 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:31:26.825 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:31:27.099 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:31:27.107 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:31:27.404 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:31:27.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:31:27.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:31:27.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:31:27.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:31:27.413 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:27.413 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:27.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:27.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:27.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:27.522 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:27.523 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:31:27.523 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:31:27.524 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:31:27.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:31:27.524 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:31:27.525 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:31:27.525 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:27.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:31:27.526 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:31:27.526 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:27.527 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:27.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:31:27.528 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:31:27.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:31:27.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:27.538 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:27.539 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:31:27.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:31:27.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:31:27.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T16:31:27.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T16:31:27.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T16:31:27.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T16:31:27.541 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:31:27.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:31:27.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:31:27.542 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:27.542 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:27.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:27.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:27.542 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:27.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:27.543 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:31:27.543 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:31:27.544 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:31:27.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:31:27.545 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:31:27.545 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:31:27.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:27.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:31:27.546 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:31:27.547 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:27.547 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:27.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:31:27.548 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:31:27.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:31:27.549 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:27.550 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:27.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:31:27.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:31:27.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:31:27.550 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T16:31:27.552 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T16:31:27.552 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:27.552 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:27.552 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.95545 2026-02-19T16:31:27.555 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T16:31:27.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T16:31:27.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T16:31:27.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T16:31:27.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T16:31:27.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T16:31:27.555 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T16:31:27.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T16:31:27.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T16:31:27.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T16:31:27.556 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T16:31:27.573 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T16:31:27.573 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:27.573 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:27.574 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:27.574 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:27.574 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:27.574 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:27.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T16:31:27.597 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T16:31:27.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T16:31:27.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:31:27.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:31:27.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T16:31:27.598 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T16:31:27.598 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:31:27.599 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get fsid 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:31:27.643 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:31:27.644 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:27.644 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:27.644 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:27.644 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:31:27.644 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T16:31:27.644 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.95545/ceph-mon.a.asok config get mon_host 2026-02-19T16:31:27.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T16:31:27.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T16:31:27.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T16:31:27.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T16:31:27.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T16:31:27.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T16:31:27.687 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T16:31:27.789 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T16:31:27.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:27.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:27.790 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:27.790 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:27.790 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:27.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:27.790 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:31:27.791 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T16:31:27.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T16:31:27.806 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T16:31:27.899 INFO:tasks.workunit.client.0.trial127.stderr:pool 'rbd' already exists 2026-02-19T16:31:27.905 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:31:28.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T16:31:28.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T16:31:28.907 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T16:31:28.907 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:28.907 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:28.907 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:28.908 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-mon.a.asok 2026-02-19T16:31:28.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T16:31:28.908 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-mon.a.asok log flush 2026-02-19T16:31:28.947 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:31:28.951 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T16:31:28.952 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:31:27.588+0000 7f9e73890c00 0 load: jerasure load: lrc 2026-02-19T16:31:28.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-19T16:31:28.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-19T16:31:28.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-19T16:31:28.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-19T16:31:28.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T16:31:28.953 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T16:31:28.953 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T16:31:28.954 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T16:31:28.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:28.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T16:31:28.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:28.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:28.955 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:28.956 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:28.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T16:31:28.958 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:28.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7a0752b8-65b1-43bd-b6fa-1d8ff9806307 2026-02-19T16:31:28.959 INFO:tasks.workunit.client.0.trial127.stdout:add osd0 7a0752b8-65b1-43bd-b6fa-1d8ff9806307 2026-02-19T16:31:28.959 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7a0752b8-65b1-43bd-b6fa-1d8ff9806307' 2026-02-19T16:31:28.959 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:28.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgOpdpnWCqORAAIneYtXaUTp4EoCIomjM8AA== 2026-02-19T16:31:28.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgOpdpnWCqORAAIneYtXaUTp4EoCIomjM8AA=="}' 2026-02-19T16:31:28.968 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7a0752b8-65b1-43bd-b6fa-1d8ff9806307 -i td/test-erasure-eio/0/new.json 2026-02-19T16:31:29.073 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:29.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T16:31:29.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgOpdpnWCqORAAIneYtXaUTp4EoCIomjM8AA== --osd-uuid 7a0752b8-65b1-43bd-b6fa-1d8ff9806307 2026-02-19T16:31:29.097 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.095+0000 7f7e4d810780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:29.099 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.097+0000 7f7e4d810780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:29.101 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.098+0000 7f7e4d810780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:29.101 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.098+0000 7f7e4d810780 -1 bdev(0x2e84c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:29.101 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.098+0000 7f7e4d810780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T16:31:29.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T16:31:29.574 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:29.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T16:31:29.575 INFO:tasks.workunit.client.0.trial127.stdout:adding osd0 key to auth repository 2026-02-19T16:31:29.575 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:29.680 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T16:31:29.681 INFO:tasks.workunit.client.0.trial127.stdout:start osd.0 2026-02-19T16:31:29.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:29.681 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:29.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:29.682 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:29.695 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.693+0000 7fd815814780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:29.697 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.694+0000 7fd815814780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:29.698 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.695+0000 7fd815814780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:29.776 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T16:31:29.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:29.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T16:31:29.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:29.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:29.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:29.777 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:29.778 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:29.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:29.778 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:31:29.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:29.903 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:29.901+0000 7fd815814780 -1 Falling back to public interface 2026-02-19T16:31:30.170 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:30.168+0000 7fd815814780 -1 osd.0 0 log_to_monitors true 2026-02-19T16:31:30.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:30.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:30.873 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:30.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:30.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:30.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T16:31:31.100 INFO:tasks.workunit.client.0.trial127.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1844757040,v1:127.0.0.1:6803/1844757040] [v2:127.0.0.1:6804/1844757040,v1:127.0.0.1:6805/1844757040] exists,up 7a0752b8-65b1-43bd-b6fa-1d8ff9806307 2026-02-19T16:31:31.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:31.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:31.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:31.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:31.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T16:31:31.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:31.104 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:31.105 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T16:31:31.106 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:31.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a2db051c-6685-48cc-b61a-804bfc6ba6e1 2026-02-19T16:31:31.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a2db051c-6685-48cc-b61a-804bfc6ba6e1' 2026-02-19T16:31:31.106 INFO:tasks.workunit.client.0.trial127.stdout:add osd1 a2db051c-6685-48cc-b61a-804bfc6ba6e1 2026-02-19T16:31:31.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:31.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDjOpdp3jHNBhAAYS6H+sdzF2er7vPHiNvHOw== 2026-02-19T16:31:31.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDjOpdp3jHNBhAAYS6H+sdzF2er7vPHiNvHOw=="}' 2026-02-19T16:31:31.116 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a2db051c-6685-48cc-b61a-804bfc6ba6e1 -i td/test-erasure-eio/1/new.json 2026-02-19T16:31:31.346 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:31.354 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T16:31:31.355 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDjOpdp3jHNBhAAYS6H+sdzF2er7vPHiNvHOw== --osd-uuid a2db051c-6685-48cc-b61a-804bfc6ba6e1 2026-02-19T16:31:31.372 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:31.369+0000 7f3e47829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:31.373 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:31.370+0000 7f3e47829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:31.375 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:31.371+0000 7f3e47829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:31.375 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:31.372+0000 7f3e47829780 -1 bdev(0x2abcc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:31.375 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:31.372+0000 7f3e47829780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T16:31:31.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T16:31:31.692 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:31.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T16:31:31.693 INFO:tasks.workunit.client.0.trial127.stdout:adding osd1 key to auth repository 2026-02-19T16:31:31.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:32.006 INFO:tasks.workunit.client.0.trial127.stdout:start osd.1 2026-02-19T16:31:32.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T16:31:32.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:32.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:32.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:32.007 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:32.021 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:32.019+0000 7f9ed8236780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:32.023 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:32.020+0000 7f9ed8236780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:32.023 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:32.021+0000 7f9ed8236780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:32.164 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:32.162+0000 7f9ed8236780 -1 Falling back to public interface 2026-02-19T16:31:32.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T16:31:32.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:32.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T16:31:32.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:32.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:32.234 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:32.235 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:32.235 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:32.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:32.236 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:31:32.464 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:32.513 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:32.511+0000 7f9ed8236780 -1 osd.1 0 log_to_monitors true 2026-02-19T16:31:33.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:33.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:33.465 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:33.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:33.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:33.465 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T16:31:33.693 INFO:tasks.workunit.client.0.trial127.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3001484457,v1:127.0.0.1:6811/3001484457] [v2:127.0.0.1:6812/3001484457,v1:127.0.0.1:6813/3001484457] exists,up a2db051c-6685-48cc-b61a-804bfc6ba6e1 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:33.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:33.695 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:31:33.696 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:33.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=05d18a0f-d473-4e4c-97b2-36aa181ac485 2026-02-19T16:31:33.697 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 05d18a0f-d473-4e4c-97b2-36aa181ac485' 2026-02-19T16:31:33.697 INFO:tasks.workunit.client.0.trial127.stdout:add osd2 05d18a0f-d473-4e4c-97b2-36aa181ac485 2026-02-19T16:31:33.697 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:33.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDlOpdpf9oZKhAAtzm6DVvWjx4i0ScIPcM6VA== 2026-02-19T16:31:33.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDlOpdpf9oZKhAAtzm6DVvWjx4i0ScIPcM6VA=="}' 2026-02-19T16:31:33.707 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 05d18a0f-d473-4e4c-97b2-36aa181ac485 -i td/test-erasure-eio/2/new.json 2026-02-19T16:31:33.937 INFO:tasks.workunit.client.0.trial127.stdout:2 2026-02-19T16:31:33.945 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T16:31:33.946 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlOpdpf9oZKhAAtzm6DVvWjx4i0ScIPcM6VA== --osd-uuid 05d18a0f-d473-4e4c-97b2-36aa181ac485 2026-02-19T16:31:33.962 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:33.960+0000 7f7a76a12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:33.963 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:33.961+0000 7f7a76a12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:33.964 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:33.962+0000 7f7a76a12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:33.965 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:33.962+0000 7f7a76a12780 -1 bdev(0x43acc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:33.965 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:33.962+0000 7f7a76a12780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T16:31:34.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T16:31:34.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:34.286 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T16:31:34.287 INFO:tasks.workunit.client.0.trial127.stdout:adding osd2 key to auth repository 2026-02-19T16:31:34.287 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:34.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T16:31:34.603 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:31:34.603 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:34.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:34.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:34.604 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:34.618 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:34.615+0000 7f6d0d24b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:34.619 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:34.617+0000 7f6d0d24b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:34.620 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:34.617+0000 7f6d0d24b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:34.755 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:34.752+0000 7f6d0d24b780 -1 Falling back to public interface 2026-02-19T16:31:34.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T16:31:34.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:34.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:31:34.835 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:34.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:34.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:34.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:34.836 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:34.836 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:34.837 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:35.068 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:35.071 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:35.069+0000 7f6d0d24b780 -1 osd.2 0 log_to_monitors true 2026-02-19T16:31:36.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:36.069 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:36.070 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:36.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:36.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:36.070 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:36.301 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2193140844,v1:127.0.0.1:6819/2193140844] [v2:127.0.0.1:6820/2193140844,v1:127.0.0.1:6821/2193140844] exists,up 05d18a0f-d473-4e4c-97b2-36aa181ac485 2026-02-19T16:31:36.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:36.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:36.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:36.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T16:31:36.305 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T16:31:36.306 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:36.307 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T16:31:36.308 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4bf71c02-c804-4562-931d-560788613249 2026-02-19T16:31:36.309 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 4bf71c02-c804-4562-931d-560788613249' 2026-02-19T16:31:36.309 INFO:tasks.workunit.client.0.trial127.stdout:add osd3 4bf71c02-c804-4562-931d-560788613249 2026-02-19T16:31:36.309 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T16:31:36.315 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDoOpdpIy64EhAABAb5jwcnzcgunoz7QN4V1Q== 2026-02-19T16:31:36.316 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDoOpdpIy64EhAABAb5jwcnzcgunoz7QN4V1Q=="}' 2026-02-19T16:31:36.316 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4bf71c02-c804-4562-931d-560788613249 -i td/test-erasure-eio/3/new.json 2026-02-19T16:31:36.545 INFO:tasks.workunit.client.0.trial127.stdout:3 2026-02-19T16:31:36.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T16:31:36.553 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDoOpdpIy64EhAABAb5jwcnzcgunoz7QN4V1Q== --osd-uuid 4bf71c02-c804-4562-931d-560788613249 2026-02-19T16:31:36.570 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:36.568+0000 7ff64de33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:36.571 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:36.569+0000 7ff64de33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:36.572 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:36.570+0000 7ff64de33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:36.572 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:36.570+0000 7ff64de33780 -1 bdev(0x298cc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T16:31:36.572 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:36.571+0000 7ff64de33780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T16:31:36.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T16:31:36.894 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T16:31:36.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T16:31:36.895 INFO:tasks.workunit.client.0.trial127.stdout:adding osd3 key to auth repository 2026-02-19T16:31:36.895 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T16:31:37.224 INFO:tasks.workunit.client.0.trial127.stdout:start osd.3 2026-02-19T16:31:37.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T16:31:37.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:37.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T16:31:37.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T16:31:37.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T16:31:37.240 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:37.238+0000 7fcbb7e30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:37.241 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:37.240+0000 7fcbb7e30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:37.242 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:37.241+0000 7fcbb7e30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:37.378 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:37.376+0000 7fcbb7e30780 -1 Falling back to public interface 2026-02-19T16:31:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T16:31:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T16:31:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:37.449 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:37.450 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:37.450 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:31:37.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:37.723 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:37.721+0000 7fcbb7e30780 -1 osd.3 0 log_to_monitors true 2026-02-19T16:31:38.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:38.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:38.675 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:38.675 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:38.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:38.676 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T16:31:38.902 INFO:tasks.workunit.client.0.trial127.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3737648772,v1:127.0.0.1:6827/3737648772] [v2:127.0.0.1:6828/3737648772,v1:127.0.0.1:6829/3737648772] exists,up 4bf71c02-c804-4562-931d-560788613249 2026-02-19T16:31:38.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:38.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:38.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.0.asok 2026-02-19T16:31:38.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T16:31:38.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.0.asok log flush 2026-02-19T16:31:38.950 INFO:tasks.workunit.client.0.trial127.stdout:{} 2026-02-19T16:31:38.954 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T16:31:38.956 INFO:tasks.workunit.client.0.trial127.stdout:2026-02-19T16:31:29.903+0000 7fd815814780 0 load: jerasure load: lrc 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T16:31:38.957 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T16:31:39.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T16:31:39.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T16:31:39.622 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' already exists 2026-02-19T16:31:39.631 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T16:31:40.633 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T16:31:40.633 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:31:40.633 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:31:40.633 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:31:40.633 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:31:40.633 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:31:40.633 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:31:40.634 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:31:40.634 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:31:40.634 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:31:40.722 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:31:40.722 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:31:40.722 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:31:40.722 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:31:40.722 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:31:40.722 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:31:40.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:31:40.950 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:40.950 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:31:40.950 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:31:40.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:31:40.950 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:40.951 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:31:41.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-19T16:31:41.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-19T16:31:41.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-19T16:31:41.016 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:41.017 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:31:41.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-19T16:31:41.080 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-19T16:31:41.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-19T16:31:41.081 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:41.081 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:31:41.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T16:31:41.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T16:31:41.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331' 2026-02-19T16:31:41.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:41.147 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:31:41.209 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T16:31:41.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T16:31:41.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331 3-111669149699' 2026-02-19T16:31:41.210 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:41.210 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-19T16:31:41.210 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:41.211 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:31:41.211 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-19T16:31:41.211 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:41.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-19T16:31:41.212 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-19T16:31:41.213 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803780 2026-02-19T16:31:41.213 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:41.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-19T16:31:41.438 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:31:42.439 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:31:42.439 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:42.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-19T16:31:42.671 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:42.671 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-19T16:31:42.671 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:42.672 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:31:42.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-19T16:31:42.673 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:42.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-19T16:31:42.674 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-19T16:31:42.674 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607556 2026-02-19T16:31:42.675 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:31:42.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-19T16:31:42.902 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:42.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-19T16:31:42.902 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:42.903 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:31:42.903 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T16:31:42.904 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:42.904 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-19T16:31:42.905 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T16:31:42.905 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 77309411331 2026-02-19T16:31:42.905 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:31:43.128 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T16:31:43.128 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:43.129 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-19T16:31:43.129 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:43.130 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:31:43.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T16:31:43.130 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:43.131 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-19T16:31:43.132 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149699 2026-02-19T16:31:43.132 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T16:31:43.132 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:31:43.361 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-19T16:31:43.362 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:31:43.363 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:43.363 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:43.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:31:43.685 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:31:43.685 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:31:43.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:31:43.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:31:43.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:31:43.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:31:43.686 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:31:43.909 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:31:43.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:31:43.910 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:43.911 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-95545-1 2026-02-19T16:31:44.225 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-95545-1 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-95545-1 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T16:31:44.226 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-95545-1 td/test-erasure-eio/ORIGINAL 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-95545-1 td/test-erasure-eio 1 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-95545-1 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-95545-1 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-1 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-1 2026-02-19T16:31:44.244 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:31:44.478 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:31:44.478 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:44.478 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:31:44.478 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:31:44.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T16:31:44.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-19T16:31:44.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-19T16:31:44.479 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-19T16:31:44.479 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T16:31:44.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-19T16:31:44.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T16:31:44.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T16:31:44.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T16:31:44.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T16:31:44.480 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T16:31:44.480 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T16:31:44.481 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T16:31:44.481 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:31:44.481 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:31:44.481 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:44.481 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:44.481 INFO:tasks.workunit.client.0.trial127.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:44.481 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:31:44.482 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.95545/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T16:31:44.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T16:31:44.529 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-19T16:31:44.530 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-19T16:31:44.530 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.95545/ceph-osd.1.asok 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-19T16:31:44.531 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.95545/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-95545-1 1 2026-02-19T16:31:44.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-95545-1 2026-02-19T16:31:44.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T16:31:44.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T16:31:44.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-95545-1 2026-02-19T16:31:44.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T16:31:44.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T16:31:44.577 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-95545-1 td/test-erasure-eio/COPY 2026-02-19T16:31:44.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T16:31:44.595 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T16:31:44.596 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-19T16:31:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-95545-1 2026-02-19T16:31:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:31:44.596 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-1 2026-02-19T16:31:44.597 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-1 2026-02-19T16:31:44.597 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:31:44.828 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:31:44.828 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:44.828 INFO:tasks.workunit.client.0.trial127.stderr:2' 2026-02-19T16:31:44.828 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:44.829 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:45.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:45.135 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-19T16:31:45.413 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already out. 2026-02-19T16:31:45.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-95545-1 2026-02-19T16:31:45.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-19T16:31:45.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T16:31:45.421 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-95545-1 2026-02-19T16:31:45.422 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-95545-1 2026-02-19T16:31:45.422 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T16:31:45.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T16:31:45.658 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:45.658 INFO:tasks.workunit.client.0.trial127.stderr:0' 2026-02-19T16:31:45.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-19T16:31:45.658 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-19T16:31:45.905 INFO:tasks.workunit.client.0.trial127.stderr:osd.2 is already in. 2026-02-19T16:31:45.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-19T16:31:45.915 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T16:31:45.916 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T16:31:45.917 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T16:31:45.918 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T16:31:45.918 INFO:tasks.workunit.client.0.trial127.stdout:start osd.2 2026-02-19T16:31:45.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=8e90201e-f8d6-4031-a9a4-17a3313aacb1 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.95545/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T16:31:45.919 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T16:31:45.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T16:31:45.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T16:31:45.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T16:31:45.919 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T16:31:45.931 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:45.930+0000 7f0067052780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:45.932 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:45.931+0000 7f0067052780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:45.933 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:45.932+0000 7f0067052780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T16:31:46.090 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:46.088+0000 7f0067052780 -1 Falling back to public interface 2026-02-19T16:31:46.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T16:31:46.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T16:31:46.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T16:31:46.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T16:31:46.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T16:31:46.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:46.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T16:31:46.148 INFO:tasks.workunit.client.0.trial127.stdout:0 2026-02-19T16:31:46.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:46.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:46.373 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T16:31:46.392 INFO:tasks.workunit.client.0.trial127.stderr:2026-02-19T16:31:46.390+0000 7f0067052780 -1 osd.2 34 log_to_monitors true 2026-02-19T16:31:47.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T16:31:47.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T16:31:47.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T16:31:47.374 INFO:tasks.workunit.client.0.trial127.stdout:1 2026-02-19T16:31:47.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T16:31:47.374 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T16:31:47.603 INFO:tasks.workunit.client.0.trial127.stdout:osd.2 up in weight 1 up_from 43 up_thru 19 down_at 35 last_clean_interval [18,34) [v2:127.0.0.1:6818/1202613518,v1:127.0.0.1:6819/1202613518] [v2:127.0.0.1:6820/1202613518,v1:127.0.0.1:6821/1202613518] exists,up 05d18a0f-d473-4e4c-97b2-36aa181ac485 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T16:31:47.605 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T16:31:47.606 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T16:31:47.606 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T16:31:47.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T16:31:47.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T16:31:47.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T16:31:47.693 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T16:31:47.694 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T16:31:47.694 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T16:31:47.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T16:31:47.921 INFO:tasks.workunit.client.0.trial127.stderr:1 2026-02-19T16:31:47.921 INFO:tasks.workunit.client.0.trial127.stderr:2 2026-02-19T16:31:47.921 INFO:tasks.workunit.client.0.trial127.stderr:3' 2026-02-19T16:31:47.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T16:31:47.921 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:47.921 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-19T16:31:47.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T16:31:47.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T16:31:47.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T16:31:47.986 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:47.986 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-19T16:31:48.050 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-19T16:31:48.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-19T16:31:48.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-19T16:31:48.051 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:48.051 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-19T16:31:48.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-19T16:31:48.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-19T16:31:48.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-184683593731' 2026-02-19T16:31:48.115 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T16:31:48.115 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-19T16:31:48.180 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T16:31:48.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T16:31:48.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-184683593731 3-111669149702' 2026-02-19T16:31:48.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:48.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-19T16:31:48.181 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:48.181 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-19T16:31:48.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T16:31:48.182 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:48.183 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-19T16:31:48.184 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.0 seq 25769803784 2026-02-19T16:31:48.184 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T16:31:48.184 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:48.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-19T16:31:48.412 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:31:49.413 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T16:31:49.413 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:49.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-19T16:31:49.640 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-19T16:31:50.641 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T16:31:50.642 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T16:31:50.872 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T16:31:50.873 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:50.873 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607559 2026-02-19T16:31:50.873 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:50.874 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-19T16:31:50.875 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-19T16:31:50.875 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:50.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607559 2026-02-19T16:31:50.876 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.1 seq 51539607559 2026-02-19T16:31:50.876 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-19T16:31:50.876 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T16:31:51.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607560 -lt 51539607559 2026-02-19T16:31:51.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:51.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-184683593731 2026-02-19T16:31:51.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:51.104 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-19T16:31:51.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-19T16:31:51.105 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:51.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593731 2026-02-19T16:31:51.106 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-19T16:31:51.106 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.2 seq 184683593731 2026-02-19T16:31:51.107 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T16:31:51.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-19T16:31:51.332 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-19T16:31:51.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-19T16:31:51.332 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-19T16:31:51.333 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-19T16:31:51.334 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T16:31:51.334 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-19T16:31:51.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-19T16:31:51.335 INFO:tasks.workunit.client.0.trial127.stdout:waiting osd.3 seq 111669149702 2026-02-19T16:31:51.335 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T16:31:51.335 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T16:31:51.568 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149702 2026-02-19T16:31:51.569 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T16:31:51.569 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:51.569 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:51.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T16:31:51.881 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T16:31:51.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T16:31:51.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T16:31:51.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T16:31:51.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T16:31:51.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T16:31:51.882 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T16:31:52.103 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T16:31:52.103 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T16:31:52.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T16:31:52.104 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T16:31:52.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T16:31:52.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T16:31:52.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T16:31:52.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-19T16:31:52.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T16:31:52.418 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T16:31:52.692 INFO:tasks.workunit.client.0.trial127.stderr:pool 'pool-jerasure' does not exist 2026-02-19T16:31:52.700 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T16:31:52.999 INFO:tasks.workunit.client.0.trial127.stderr:erasure-code-profile myprofile does not exist 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:53.009 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:53.118 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:53.118 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:31:53.119 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:31:53.120 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:31:53.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:31:53.121 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:31:53.122 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:31:53.122 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:53.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:31:53.123 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:31:53.123 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:53.124 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:53.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T16:31:53.125 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:31:53.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:31:53.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:53.136 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:53.136 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:31:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:31:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:31:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-19T16:31:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-eio 0 2026-02-19T16:31:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T16:31:53.138 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T16:31:53.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T16:31:53.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T16:31:53.139 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T16:31:53.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T16:31:53.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T16:31:53.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T16:31:53.139 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T16:31:53.140 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T16:31:53.140 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T16:31:53.141 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T16:31:53.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T16:31:53.142 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T16:31:53.142 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T16:31:53.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:53.143 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T16:31:53.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T16:31:53.144 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T16:31:53.144 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:53.145 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T16:31:53.146 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T16:31:53.147 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T16:31:53.147 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T16:31:53.147 INFO:tasks.workunit.client.0.trial127.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.95545 2026-02-19T16:31:53.147 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.95545 2026-02-19T16:31:53.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T16:31:53.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T16:31:53.148 INFO:tasks.workunit.client.0.trial127.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-19T16:31:53.149 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T16:31:53.149 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T16:31:53.216 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-19T16:31:53.216 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-19T16:31:53.576 DEBUG:teuthology.parallel:result is None 2026-02-19T16:31:53.576 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T16:31:53.601 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T16:31:53.602 DEBUG:teuthology.orchestra.run.trial127:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-19T16:31:53.658 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T16:31:53.658 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-19T16:31:53.673 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-19T16:31:53.673 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-19T16:31:53.814 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-19T16:31:53.826 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-19T16:31:53.826 DEBUG:teuthology.orchestra.run.trial127:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T16:31:53.842 INFO:teuthology.orchestra.run.trial127.stderr:bash: line 1: ntpq: command not found 2026-02-19T16:31:53.951 INFO:teuthology.orchestra.run.trial127.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T16:31:53.951 INFO:teuthology.orchestra.run.trial127.stdout:=============================================================================== 2026-02-19T16:31:53.952 INFO:teuthology.orchestra.run.trial127.stdout:^- 149.28.200.179.vultruser> 2 6 377 29 -599us[ -599us] +/- 41ms 2026-02-19T16:31:53.952 INFO:teuthology.orchestra.run.trial127.stdout:^+ ntp.netlinkify.com 2 6 377 29 -416us[ -416us] +/- 7149us 2026-02-19T16:31:53.952 INFO:teuthology.orchestra.run.trial127.stdout:^* 38.70.62.230 1 6 377 96 +496us[ +483us] +/- 6788us 2026-02-19T16:31:53.952 INFO:teuthology.orchestra.run.trial127.stdout:^+ 144.202.62.209.vultruser> 3 6 377 225 -811us[-1152us] +/- 21ms 2026-02-19T16:31:53.953 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-19T16:31:53.964 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-19T16:31:53.965 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-19T16:31:53.981 DEBUG:teuthology.orchestra.run.trial127:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-19T16:31:54.046 DEBUG:teuthology.orchestra.run.trial127:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-19T16:31:54.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T16:31:54.071 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-19T16:31:54.083 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-19T16:31:54.096 INFO:teuthology.task.internal:Duration was 1103.496621 seconds 2026-02-19T16:31:54.097 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-19T16:31:54.115 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-19T16:31:54.115 DEBUG:teuthology.orchestra.run.trial127:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-19T16:31:54.152 INFO:teuthology.orchestra.run.trial127.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T16:31:54.338 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-19T16:31:54.338 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial127.front.sepia.ceph.com 2026-02-19T16:31:54.338 DEBUG:teuthology.orchestra.run.trial127:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-19T16:31:54.396 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-19T16:31:54.396 DEBUG:teuthology.orchestra.run.trial127:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T16:31:54.497 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-19T16:31:54.497 DEBUG:teuthology.orchestra.run.trial127:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-19T16:31:54.522 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T16:31:54.522 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T16:31:54.522 INFO:teuthology.orchestra.run.trial127.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-19T16:31:54.522 INFO:teuthology.orchestra.run.trial127.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T16:31:54.522 INFO:teuthology.orchestra.run.trial127.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-19T16:31:54.532 INFO:teuthology.orchestra.run.trial127.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-19T16:31:54.534 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-19T16:31:54.550 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-19T16:31:54.550 DEBUG:teuthology.orchestra.run.trial127:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-19T16:31:54.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-19T16:31:54.615 DEBUG:teuthology.orchestra.run.trial127:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:54.660 INFO:teuthology.orchestra.run.trial127.stdout:kernel.core_pattern = core 2026-02-19T16:31:54.674 DEBUG:teuthology.orchestra.run.trial127:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-19T16:31:54.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T16:31:54.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-19T16:31:54.746 INFO:teuthology.task.internal:Transferring archived files... 2026-02-19T16:31:54.749 DEBUG:teuthology.misc:Transferring archived files from trial127:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59050/remote/trial127 2026-02-19T16:31:54.750 DEBUG:teuthology.orchestra.run.trial127:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-19T16:31:54.806 INFO:teuthology.task.internal:Removing archive directory... 2026-02-19T16:31:54.806 DEBUG:teuthology.orchestra.run.trial127:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-19T16:31:54.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-19T16:31:54.876 INFO:teuthology.task.internal:Not uploading archives. 2026-02-19T16:31:54.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-19T16:31:54.895 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-19T16:31:54.895 DEBUG:teuthology.orchestra.run.trial127:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-19T16:31:54.911 INFO:teuthology.orchestra.run.trial127.stdout: 917572 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 16:31 /home/ubuntu/cephtest 2026-02-19T16:31:54.912 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-19T16:31:54.931 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-19T16:31:54.969 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} duration: 1103.4966213703156 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-19T16:31:54.970 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T16:31:55.087 INFO:teuthology.run:pass